Network Working Group Y. Gu Internet-Draft H. Song Intended status: Informational Huawei Expires: June 20, 2010 Y. Yang R. Alimi Yale University December 17, 2009 DECADE Requirements draft-gu-decade-reqs-02 Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on June 20, 2010. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Gu, et al. Expires June 20, 2010 [Page 1] Internet-Draft DECADE Requirements December 2009 Abstract The target of DECoupled Application Data Enroute (DECADE) is to specify a protocol that can be used by a P2P application to utilize storage and related resources in an in-network storage element. This document enumerates and explains requirements that should be considered during the design and implementation of this protocol. Gu, et al. Expires June 20, 2010 [Page 2] Internet-Draft DECADE Requirements December 2009 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Terminology and Concepts . . . . . . . . . . . . . . . . . . . 4 3. General Principles . . . . . . . . . . . . . . . . . . . . . . 4 3.1. Client Simplicity . . . . . . . . . . . . . . . . . . . . 4 3.2. Storage Management . . . . . . . . . . . . . . . . . . . . 5 3.3. Access by Other Users . . . . . . . . . . . . . . . . . . 5 3.4. Resource Control . . . . . . . . . . . . . . . . . . . . . 5 3.5. Data Object Size . . . . . . . . . . . . . . . . . . . . . 6 4. Data Access . . . . . . . . . . . . . . . . . . . . . . . . . 7 4.1. Reading/Writing Own Storage . . . . . . . . . . . . . . . 7 4.2. Allow Others to Read/Write Storage . . . . . . . . . . . . 7 4.3. Communication among In-network Storage Elements . . . . . 7 5. Data Management . . . . . . . . . . . . . . . . . . . . . . . 8 5.1. Explicit Deletion of Stored Data . . . . . . . . . . . . . 8 5.2. Time-to-live for Stored Data . . . . . . . . . . . . . . . 8 5.3. Storage Status . . . . . . . . . . . . . . . . . . . . . . 8 6. Resource Control . . . . . . . . . . . . . . . . . . . . . . . 9 6.1. Multiple Applications . . . . . . . . . . . . . . . . . . 9 6.2. Per-Peer, Per-Data Control . . . . . . . . . . . . . . . . 9 6.3. Server Involvement . . . . . . . . . . . . . . . . . . . . 10 7. Authorization . . . . . . . . . . . . . . . . . . . . . . . . 10 7.1. Per-Peer, Per-Data Read Access . . . . . . . . . . . . . . 10 7.2. Per-User Write Access . . . . . . . . . . . . . . . . . . 11 7.3. Authorization Checks . . . . . . . . . . . . . . . . . . . 11 7.4. Server Involvement . . . . . . . . . . . . . . . . . . . . 12 8. Data Availability . . . . . . . . . . . . . . . . . . . . . . 12 8.1. Offline Usage . . . . . . . . . . . . . . . . . . . . . . 12 9. Error Conditions . . . . . . . . . . . . . . . . . . . . . . . 12 9.1. Insufficient Resources . . . . . . . . . . . . . . . . . . 12 9.2. Unavailable and Deleted Data . . . . . . . . . . . . . . . 13 9.3. Overload Conditions . . . . . . . . . . . . . . . . . . . 13 10. Protocol Applicability . . . . . . . . . . . . . . . . . . . . 13 10.1. NATs and Firewalls . . . . . . . . . . . . . . . . . . . . 13 10.2. Connections to Clients . . . . . . . . . . . . . . . . . . 14 11. Security Considerations . . . . . . . . . . . . . . . . . . . 14 12. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 14 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 15 15. References . . . . . . . . . . . . . . . . . . . . . . . . . . 15 15.1. Normative References . . . . . . . . . . . . . . . . . . . 15 15.2. Informative References . . . . . . . . . . . . . . . . . . 15 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 15 Gu, et al. Expires June 20, 2010 [Page 3] Internet-Draft DECADE Requirements December 2009 1. Introduction The goal of DECADE is to specify a protocol that allows P2P applications to use and control storage and related resources in an in-network storage element. In this context, using storage resources includes basic capabilies such as storing and retrieving data, but also (1) controlling access by peers with which it is sharing data and (2) controlling the resources used by remote peers to access that data. This document enumerates and explains the rationale behind requirements that should be considered in the specification of a protocol for DECADE. More details about DECADE can be found in the problem statement [I-D.song-decade-problem-statement]. This document will be updated to track revisions to the problem statement. 2. Terminology and Concepts The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. This document uses terms defined in [I-D.song-decade-problem-statement]. In particular, IAP refers to the In-network storage Access Protocol, which is the protocol used to communicate between a DECADE client and DECADE server (in-network storage). 3. General Principles A number of general principles are identified in this section. These general principles are important in achieving the goal of DECADE to be useful in many P2P applications. 3.1. Client Simplicity 3.1.1. Requirement DECADE SHOULD be specified such that client implementations remain simple. Gu, et al. Expires June 20, 2010 [Page 4] Internet-Draft DECADE Requirements December 2009 3.1.2. Rationale The P2P paradigm has been applied in many contexts such as file- sharing, streaming, and video-on-demand, resulting in the development of a large amount of software supporting such systems. It is envisioned that many applications can benefit from supporting DECADE, so supporting the specified protocol should be simple for client applications. 3.2. Storage Management 3.2.1. Requirement DECADE MUST support the ability for a user to own and manage storage space at in-network storage element. 3.2.2. Rationale A primary departure from an existing form of in-network storage, P2P caching, is the ability for applications to manage the data stored. This allows applications to persist data in the network without fear that it may be deleted (e.g., pushed out of the cache) by other users. 3.3. Access by Other Users 3.3.1. Requirement DECADE MUST support the ability for a user to apply access control policies to users other than itself for its storage. The users with whom access is being shared can be under a different administrative domain than the user who owns the in-network storage. 3.3.2. Rationale Peers in a P2P application may be located across multiple ISPs under multiple administrative domains. Thus, to be useful by P2P applications, DECADE allows a user to specify access control policies for users that may or may not be known to the user's storage provider. 3.4. Resource Control 3.4.1. Requirement DECADE MUST provide the ability for a user to define resource usage guidelines for resources consumed by other users when accessing its storage. A DECADE client MUST be able to control parameters for at Gu, et al. Expires June 20, 2010 [Page 5] Internet-Draft DECADE Requirements December 2009 least the following types of resources: o Bandwidth (relative weight) o Storage space o Network connections 3.4.2. Rationale Recent developments in P2P systems have rely on resource-control to achieve certain application-level goals, such as improving incentives [LLSB08] and forming distribution topologies [PPLive]. It is important that DECADE provides basic support for applications to meet such requirements. Thus, DECADE supports more than the ability read and write data. In particular, DECADE also supports the ability to control resources used by remote peers while accessing data in in- network storage. There can multiple options when controlling bandwidth used by remote peers. We indicate that a relative weight can be used to share bandwidth amongst multiple external peers. Other possibilities (not exclusive of using a relative weight) may be to also specify a maximum bandwidth, or indicating an order in which requests should be served (as is done in eMule). However, care should be taken to not require complex implementation by the server. 3.5. Data Object Size 3.5.1. Requirement DECADE MUST allow for efficient data transfer of small objects (e.g., 16KB) between a DECADE client and in-network storage with only minimal additional latency required by the protocol. 3.5.2. Rationale Though P2P applications are frequently used to share large amounts of data (e.g., continuous streams or large files), the data itself is typically subdivided into smaller chunks that are transferred between peers. Additionally, the small chunks may have requirements on delivery time (e.g., in a live-streaming application). DECADE must enable data to be efficiently transferred amongst peers at this granularity. Note that DECADE should allow for efficient data transfer for larger data objects as well. Gu, et al. Expires June 20, 2010 [Page 6] Internet-Draft DECADE Requirements December 2009 4. Data Access 4.1. Reading/Writing Own Storage 4.1.1. Requirement IAP MUST support the ability for a DECADE client to read data from and write data to its own in-network storage. 4.1.2. Rationale Two basic capabilities for any storage system are reading and writing data. A DECADE client can read data from and write data to in- network storage space that it owns. 4.2. Allow Others to Read/Write Storage 4.2.1. Requirement IAP MUST support the ability for other DECADE clients to read data from and write data to a DECADE client's own in-network storage. 4.2.2. Rationale One defining characteristic of many P2P applications is the ability to share data with peers. In particular, a client may upload data to remote peers. With DECADE, a client can indicate to a remote peer that data should be read from in-network storage instead of being uploaded directly between the peers. Similarly, in many P2P applications, a client can download data from remote peers. With DECADE, a client can indicate to a remote peer that data should be written to in-network storage instead of being downloaded directly between the peers. 4.3. Communication among In-network Storage Elements 4.3.1. Requirement DECADE SHOULD support the ability for two in-network storage elements in different administrative domains to store and/or retrieve data directly between each other. If such a protocol is supported, this MAY be the same (or a subset or extension of) as the IAP protocol. 4.3.2. Rationale Allowing server-to-server communication can reduce latency in some common scenarios. Consider a scenario when a DECADE client is Gu, et al. Expires June 20, 2010 [Page 7] Internet-Draft DECADE Requirements December 2009 downloading data into its own storage from another client's in- network storage. One possibility is for the client to first download the data itself, and then upload it to its own storage. However, this causes unnecessary latency and network traffic. Allowing the data to be downloaded from the remote in-network storage into the client's own in-network storage can alleviate both. 5. Data Management 5.1. Explicit Deletion of Stored Data 5.1.1. Requirement IAP MUST support the ability for a DECADE client to explicitly delete data from its own in-network storage. 5.1.2. Rationale A DECADE client may continually be writing data to its in-network storage. Since there may be a limit (e.g., imposed by the storage provider) to how much total storage can be used, some data may need to be removed to make room for additional data. A DECADE client should be able to explicitly remove particular data. 5.2. Time-to-live for Stored Data 5.2.1. Requirement IAP MUST support the ability for a DECADE client to indicate a time- to-live value (or expiration time) indicating a length of time until particular data is deleted by the in-network storage element. 5.2.2. Rationale Some data stored by a DECADE client may be usable only within a certain window of time, such as in live-streaming P2P applications. Providing a time-to-live value for stored data (e.g., at the time it is stored) can reduce management overhead by avoiding many 'delete' commands sent to in-network storage. 5.3. Storage Status 5.3.1. Requirement A DECADE client MUST be able to retrieve current resource usage (including list of stored data) and resource quotas on its in-network storage. Gu, et al. Expires June 20, 2010 [Page 8] Internet-Draft DECADE Requirements December 2009 5.3.2. Rationale The resources used by a client are not directly-attached (e.g., disk, network interface, etc). Thus, the client cannot locally determine how such resources are being used. Before storing and retrieving data, a client should be able to determine which data is available (e.g., after an application restart). Additionally, a client should be able to determine resource availability (see Section 3.4) to better allocate them to remote peers. 6. Resource Control 6.1. Multiple Applications 6.1.1. Requirement DECADE SHOULD support the ability for users to define resource sharing policies for multiple applications being run/managed by the user. 6.1.2. Rationale A user may own in-network storage and share the in-network storage resources amongst multiple applications. For example, the user may run a video-on-demand application and a live-streaming (or even two different live-streaming applications) application which both make use of the user's in-network storage. The applications may be running on different machines and may not directly communicate. Thus, DECADE should enable the user to determine resource sharing policies between the applications. One possibility is for a user to indicate the particular resource sharing policies between applications out-of-band (not using a standard protocol), but this requirement may manifest itself in passing values over IAP to identify individual applications. Such identifiers can be either user-generated or server-generated and do not need to be registered by IANA. 6.2. Per-Peer, Per-Data Control 6.2.1. Requirement A DECADE client MUST be able to assign resource quotas to individual peers for reading from and writing particular data to its in-network storage within a particular range of time. Gu, et al. Expires June 20, 2010 [Page 9] Internet-Draft DECADE Requirements December 2009 6.2.2. Rationale P2P applications can rely on control of resources on a per-peer or per-data basis. For example, application policy may indicate that certain peers have a higher bandwidth share for receiving data. Additionally, certain data (e.g., chunks) may be distributed with a higher priority. As another example, when allowing a remote peer to write data to a user's in-network storage, the remote peer may be restricted to write only a certain amount of data. Since the client may need to manage multiple peers accessing its data, it should be able to indicate the time over which the granted resources are usable. For example, an expiration time for the access could be indicated to the server after which no resources are granted (e.g., indicate error as access denied). 6.3. Server Involvement 6.3.1. Requirement A DECADE client MUST be able to indicate, without contacting the server itself, resource control policies for peers' requests. 6.3.2. Rationale One important consideration for in-network storage elements is scalability, since a single storage element may be used to support many users. Many P2P applications use small chunk sizes and frequent data exchanges. If such an application employed resource control and contacted the in-network storage element for each data exchange, this could present a scalability challenge for the server as well as additional latency for clients. An alternative is to pass resource control policies (in a cryptographically secure way) directly amongst peers. Peers can then present the policy to the server using IAP. This can result in reduced messaging handled by the in-network storage. 7. Authorization 7.1. Per-Peer, Per-Data Read Access 7.1.1. Requirement A DECADE Client MUST be able to authorize individual peers to read particular data stored on its in-network storage. Gu, et al. Expires June 20, 2010 [Page 10] Internet-Draft DECADE Requirements December 2009 7.1.2. Rationale A P2P application can control certain application-level policies by sending particular data (e.g., chunks) to certain peers. It is important that peers not be able to circumvent such decisions by arbitrarily reading any currently-stored data in in-network storage. 7.2. Per-User Write Access 7.2.1. Requirement A DECADE Client MUST be able to authorize individual peers to store data into its in-network storage. 7.2.2. Rationale The space managed by a user in in-network storage can be a limited resource. At the same time, it can be useful to allow remote peers to write data directly to a user's in-network storage. Thus, a DECADE client should be able to grant only certain peers this privilege. Note that it is not (currently) a requirement to check that a peer stores a particular set of data (e.g., the check that a remote peer writes the expected chunk of a file). Enforcing this as a requirement would require a client to know which data is expected (e.g., the full chunk itself or a hash of the chunk) which may not be available in all applications. Checking for a particular hash could be considered as a requirement in the future that could optionally be employed by applications. 7.3. Authorization Checks 7.3.1. Requirement In-network storage MUST check the authorization of a client before it executes a supplied request. 7.3.2. Rationale Authorization granted by a DECADE client are meaningless unless unauthorized requests are denied access. Thus, the in-network storage element must verify the authorization of a particular request before it is executed. Gu, et al. Expires June 20, 2010 [Page 11] Internet-Draft DECADE Requirements December 2009 7.4. Server Involvement 7.4.1. Requirement A DECADE client MUST be able to indicate, without contacting the server itself, access control policies for peers' requests. 7.4.2. Rationale See discussion in Section 6.3.2. 8. Data Availability 8.1. Offline Usage 8.1.1. Requirement DECADE MAY support the ability for a user to provide authorized access to its in-network storage even while no DECADE-enabled applications are currently running or connected to the network. 8.1.2. Rationale If an application desires, it can authorize peers to access its storage even after the application exits or network connectivity is lost. An example use case is mobile scenarios, where a client can lose and regain network connectivity very often. 9. Error Conditions 9.1. Insufficient Resources 9.1.1. Requirement IAP MUST support an error condition indicating to a DECADE client that resources (e.g., storage space) were not available to service a request (e.g., storage quota exceeded when attempting to store data). 9.1.2. Rationale The currently-used resource levels within the in-network storage are not locally-discoverable, since the resources (disk, network interfaces, etc) are not directly attached. In order to allocate resources appropriately amongst peers, a client must be able to determine when resource limits have been reached. The client can then respond by explicitly freeing necessary resources or waiting for Gu, et al. Expires June 20, 2010 [Page 12] Internet-Draft DECADE Requirements December 2009 such resources to be freed. 9.2. Unavailable and Deleted Data 9.2.1. Requirement IAP MUST support error conditions indicating that (1) data was rejected from being stored, (2) deleted, or (3) marked unavailable by a storage provider. 9.2.2. Rationale Storage providers may require the ability to (1) avoid storing, (2) delete, or (3) quarantine certain data that has been identified as illegal (or otherwise prohibited). DECADE does not indicate how such data is identified, but applications using DECADE should not break if a storage provider is obligated to enforce such policies. Appropriate error conditions should be indicated to applications. 9.3. Overload Conditions 9.3.1. Requirement In-network storage, which is operating close to its capacity limit (e.g., too busy servicing other requests), MUST be able to reject requests. 9.3.2. Rationale When in-network storage is operating at a limit where it may not be able to process additional requests, it should not be required to generate responses to such additional requests. Forcing the in- network storage to do so can impair its ability to service existing requests. 10. Protocol Applicability 10.1. NATs and Firewalls 10.1.1. Requirement IAP SHOULD be usable across firewalls and NATs without requiring additional network support (e.g., Application-level Gateways). Gu, et al. Expires June 20, 2010 [Page 13] Internet-Draft DECADE Requirements December 2009 10.1.2. Rationale Firewalls and NATs are widely used in the Internet today, both in ISP networks and within households. Deployment of DECADE must not require modifications to such devices (beyond, perhaps, reconfiguration). 10.2. Connections to Clients 10.2.1. Requirement IAP SHOULD NOT require that network connections be made to DECADE clients (e.g., from a server to a DECADE client or from a DECADE client to another DECADE client). 10.2.2. Rationale Many household networks and operating systems have firewalls and NATs configured by default. To ease deployment by avoiding configuration changes and help mitigate security risks, DECADE should not require clients to listen for any incoming network connections (beyond what is required by any other already-deployed application). 11. Security Considerations Authorization for access to in-network storage is an important part of the requirements listed in this document. Authorization for access to storage resources and the data itself is important for users to be able to manage and limit distribution of content. For example, a user may only wish to share particular content with certain peers. If the authorization technique implemented in DECADE passes any private information (e.g., user passwords) over the wire, it MUST be passed in a secure way. 12. Discussion Sometimes, several logical in-network storages could be deployed on the same physical network device. In this case, in-network storages on the same physical network device can communicate and transfer data through internal communication messages. However in-network storages deployed on different physical network devices SHOULD communicate with in-network storage Access Protocol (IAP). To provide fairness among users, the in-network storage provider Gu, et al. Expires June 20, 2010 [Page 14] Internet-Draft DECADE Requirements December 2009 should assign resource (e.g., storage, bandwidth, connections) quota for users. This can prevent a small number of clients from occupying large amounts of resources on the in-network storage, while others starve. 13. IANA Considerations There is no IANA consideration with this document. 14. Acknowledgments We would like to thank Reinaldo Penno, Alexey Melnikov, Rich Woundy and Ning Zong for contributions and feedback to this document. 15. References 15.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 15.2. Informative References [I-D.song-decade-problem-statement] Yongchao, S., Zong, N., Yang, Y., and R. Alimi, "DECoupled Application Data Enroute (DECADE) Problem Statement", draft-song-decade-problem-statement-00 (work in progress), October 2009. [LLSB08] Dave Levin, Katrina LaCurts, Neil Spring, Bobby Bhattacharjee., "BitTorrent is an Auction: Analyzing and Improving BitTorrent's Incentives", In SIGCOMM 2008. [PPLive] "PPLive", http://www.pplive.com. Gu, et al. Expires June 20, 2010 [Page 15] Internet-Draft DECADE Requirements December 2009 Authors' Addresses Yingjie Gu Huawei Baixia Road No. 91 Nanjing, Jiangsu Province 210001 P.R.China Phone: +86-25-84565868 Email: guyingjie@huawei.com Song Haibin Huawei Baixia Road No. 91 Nanjing, Jiangsu Province 210001 P.R.China Phone: +86-25-84565867 Email: melodysong@huawei.com Yang Richard Yang Yale University Email: yry@cs.yale.edu Richard Alimi Yale University Email: richard.alimi@yale.edu Gu, et al. Expires June 20, 2010 [Page 16]