GitHub PullRequestEvent at 2014-03-25 20:50:55
Repository: stratosphere
Actor: StephanEwen
Payload: {"action":"closed","number":511,"pull_request":{"url":"https://api.github.com/repos/stratosphere/stratosphere/pulls/511","id":12947371,"html_url":"https://github.com/stratosphere/stratosphere/pull/511","diff_url":"https://github.com/stratosphere/stratosphere/pull/511.diff","patch_url":"https://github.com/stratosphere/stratosphere/pull/511.patch","issue_url":"https://api.github.com/repos/stratosphere/stratosphere/issues/511","number":511,"state":"closed","title":"Offer buffer-oriented API for I/O (#25)","user":{"login":"uce","id":1756620,"avatar_url":"https://avatars.githubusercontent.com/u/1756620?","gravatar_id":"c4e4eb1ac5a679a212c74e8743f6ed65","url":"https://api.github.com/users/uce","html_url":"https://github.com/uce","followers_url":"https://api.github.com/users/uce/followers","following_url":"https://api.github.com/users/uce/following{/other_user}","gists_url":"https://api.github.com/users/uce/gists{/gist_id}","starred_url":"https://api.github.com/users/uce/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/uce/subscriptions","organizations_url":"https://api.github.com/users/uce/orgs","repos_url":"https://api.github.com/users/uce/repos","events_url":"https://api.github.com/users/uce/events{/privacy}","received_events_url":"https://api.github.com/users/uce/received_events","type":"User","site_admin":false},"body":"The runtime offered a record-oriented API (#25) for data transfers, which\r\n* resulted in unnecessary data (de)serialization,\r\n* complicated the upcoming fault tolerance implementation, and\r\n* blocked more efficient implementations of higher-level operators.\r\n\r\nWith this commit, the runtime offers a buffer-oriented API for the output side (sending), which is oblivious to records. The buffer oriented input side (receiving) is still to be implemented.\r\n\r\n---\r\n\r\n* Previously, the I/O classes allowed different implementations of channels, gates etc. with the `ByteBuffered*` implementation being the default one. I've merged those classes to only have the default implementation (e.g. InputChannel, OutputChannel), which greatly simplifies the code.\r\n* Every I/O class had an associated `*Context`, which was initiated at runtime for buffer allocation. All these classes could be removed, since the I/O stack does not allocate buffers anymore, but instead is fed buffers to send/receive.\r\n* Record serialization has been moved from the channels (for the output side, input side is still pending) to the writers/readers. For example `RecordWriter` still offers the record-oriented API, but serializes records into buffers and dispatches full buffers via the new buffer-oriented API (`BufferWriter`). \r\n* The (previosuly singleton) `GlobalBufferPool` (for network buffers) is now instantiated by the `ChannelManager` for each TaskManager. The network buffers are used by `RecordWriter` for record serialization and by the input channels (will be changed) for deserialization. In the future, the buffers can directly come from the upper layers (formerly PACT).\r\n* I've reorganized the touched classes into the package `eu.stratosphere.runtime.*` (previously `eu.stratosphere.nephele.*`) to start with the removal of the deprecated Nephele/PACT separation.","created_at":"2014-02-26T11:20:00Z","updated_at":"2014-03-25T20:50:55Z","closed_at":"2014-03-25T20:50:55Z","merged_at":null,"merge_commit_sha":"ba094dace7aa0b386192d2fde3f98c3d88b4e0d9","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/stratosphere/stratosphere/pulls/511/commits","review_comments_url":"https://api.github.com/repos/stratosphere/stratosphere/pulls/511/comments","review_comment_url":"https://api.github.com/repos/stratosphere/stratosphere/pulls/comments/{number}","comments_url":"https://api.github.com/repos/stratosphere/stratosphere/issues/511/comments","statuses_url":"https://api.github.com/repos/stratosphere/stratosphere/statuses/cd04a3361d95ad9f359c0846929de0dcb3fcbbc3","head":{"label":"uce:buffer_orientation","ref":"buffer_orientation","sha":"cd04a3361d95ad9f359c0846929de0dcb3fcbbc3","user":{"login":"uce","id":1756620,"avatar_url":"https://avatars.githubusercontent.com/u/1756620?","gravatar_id":"c4e4eb1ac5a679a212c74e8743f6ed65","url":"https://api.github.com/users/uce","html_url":"https://github.com/uce","followers_url":"https://api.github.com/users/uce/followers","following_url":"https://api.github.com/users/uce/following{/other_user}","gists_url":"https://api.github.com/users/uce/gists{/gist_id}","starred_url":"https://api.github.com/users/uce/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/uce/subscriptions","organizations_url":"https://api.github.com/users/uce/orgs","repos_url":"https://api.github.com/users/uce/repos","events_url":"https://api.github.com/users/uce/events{/privacy}","received_events_url":"https://api.github.com/users/uce/received_events","type":"User","site_admin":false},"repo":{"id":11394773,"name":"stratosphere","full_name":"uce/stratosphere","owner":{"login":"uce","id":1756620,"avatar_url":"https://avatars.githubusercontent.com/u/1756620?","gravatar_id":"c4e4eb1ac5a679a212c74e8743f6ed65","url":"https://api.github.com/users/uce","html_url":"https://github.com/uce","followers_url":"https://api.github.com/users/uce/followers","following_url":"https://api.github.com/users/uce/following{/other_user}","gists_url":"https://api.github.com/users/uce/gists{/gist_id}","starred_url":"https://api.github.com/users/uce/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/uce/subscriptions","organizations_url":"https://api.github.com/users/uce/orgs","repos_url":"https://api.github.com/users/uce/repos","events_url":"https://api.github.com/users/uce/events{/privacy}","received_events_url":"https://api.github.com/users/uce/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/uce/stratosphere","description":"","fork":true,"url":"https://api.github.com/repos/uce/stratosphere","forks_url":"https://api.github.com/repos/uce/stratosphere/forks","keys_url":"https://api.github.com/repos/uce/stratosphere/keys{/key_id}","collaborators_url":"https://api.github.com/repos/uce/stratosphere/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/uce/stratosphere/teams","hooks_url":"https://api.github.com/repos/uce/stratosphere/hooks","issue_events_url":"https://api.github.com/repos/uce/stratosphere/issues/events{/number}","events_url":"https://api.github.com/repos/uce/stratosphere/events","assignees_url":"https://api.github.com/repos/uce/stratosphere/assignees{/user}","branches_url":"https://api.github.com/repos/uce/stratosphere/branches{/branch}","tags_url":"https://api.github.com/repos/uce/stratosphere/tags","blobs_url":"https://api.github.com/repos/uce/stratosphere/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/uce/stratosphere/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/uce/stratosphere/git/refs{/sha}","trees_url":"https://api.github.com/repos/uce/stratosphere/git/trees{/sha}","statuses_url":"https://api.github.com/repos/uce/stratosphere/statuses/{sha}","languages_url":"https://api.github.com/repos/uce/stratosphere/languages","stargazers_url":"https://api.github.com/repos/uce/stratosphere/stargazers","contributors_url":"https://api.github.com/repos/uce/stratosphere/contributors","subscribers_url":"https://api.github.com/repos/uce/stratosphere/subscribers","subscription_url":"https://api.github.com/repos/uce/stratosphere/subscription","commits_url":"https://api.github.com/repos/uce/stratosphere/commits{/sha}","git_commits_url":"https://api.github.com/repos/uce/stratosphere/git/commits{/sha}","comments_url":"https://api.github.com/repos/uce/stratosphere/comments{/number}","issue_comment_url":"https://api.github.com/repos/uce/stratosphere/issues/comments/{number}","contents_url":"https://api.github.com/repos/uce/stratosphere/contents/{+path}","compare_url":"https://api.github.com/repos/uce/stratosphere/compare/{base}...{head}","merges_url":"https://api.github.com/repos/uce/stratosphere/merges","archive_url":"https://api.github.com/repos/uce/stratosphere/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/uce/stratosphere/downloads","issues_url":"https://api.github.com/repos/uce/stratosphere/issues{/number}","pulls_url":"https://api.github.com/repos/uce/stratosphere/pulls{/number}","milestones_url":"https://api.github.com/repos/uce/stratosphere/milestones{/number}","notifications_url":"https://api.github.com/repos/uce/stratosphere/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/uce/stratosphere/labels{/name}","releases_url":"https://api.github.com/repos/uce/stratosphere/releases{/id}","created_at":"2013-07-13T21:58:43Z","updated_at":"2014-03-24T13:21:53Z","pushed_at":"2014-03-24T13:21:52Z","git_url":"git://github.com/uce/stratosphere.git","ssh_url":"git@github.com:uce/stratosphere.git","clone_url":"https://github.com/uce/stratosphere.git","svn_url":"https://github.com/uce/stratosphere","homepage":null,"size":75710,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master","master_branch":"master"}},"base":{"label":"stratosphere:master","ref":"master","sha":"91b5a2b51a570f4e61edf301c04f18cc929accbb","user":{"login":"stratosphere","id":5637455,"avatar_url":"https://avatars.githubusercontent.com/u/5637455?","gravatar_id":"0915d6587f7c1798df4ea1ddf9fc4846","url":"https://api.github.com/users/stratosphere","html_url":"https://github.com/stratosphere","followers_url":"https://api.github.com/users/stratosphere/followers","following_url":"https://api.github.com/users/stratosphere/following{/other_user}","gists_url":"https://api.github.com/users/stratosphere/gists{/gist_id}","starred_url":"https://api.github.com/users/stratosphere/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stratosphere/subscriptions","organizations_url":"https://api.github.com/users/stratosphere/orgs","repos_url":"https://api.github.com/users/stratosphere/repos","events_url":"https://api.github.com/users/stratosphere/events{/privacy}","received_events_url":"https://api.github.com/users/stratosphere/received_events","type":"Organization","site_admin":false},"repo":{"id":9499422,"name":"stratosphere","full_name":"stratosphere/stratosphere","owner":{"login":"stratosphere","id":5637455,"avatar_url":"https://avatars.githubusercontent.com/u/5637455?","gravatar_id":"0915d6587f7c1798df4ea1ddf9fc4846","url":"https://api.github.com/users/stratosphere","html_url":"https://github.com/stratosphere","followers_url":"https://api.github.com/users/stratosphere/followers","following_url":"https://api.github.com/users/stratosphere/following{/other_user}","gists_url":"https://api.github.com/users/stratosphere/gists{/gist_id}","starred_url":"https://api.github.com/users/stratosphere/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stratosphere/subscriptions","organizations_url":"https://api.github.com/users/stratosphere/orgs","repos_url":"https://api.github.com/users/stratosphere/repos","events_url":"https://api.github.com/users/stratosphere/events{/privacy}","received_events_url":"https://api.github.com/users/stratosphere/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/stratosphere/stratosphere","description":"This is the source code repository of the Stratosphere project","fork":false,"url":"https://api.github.com/repos/stratosphere/stratosphere","forks_url":"https://api.github.com/repos/stratosphere/stratosphere/forks","keys_url":"https://api.github.com/repos/stratosphere/stratosphere/keys{/key_id}","collaborators_url":"https://api.github.com/repos/stratosphere/stratosphere/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/stratosphere/stratosphere/teams","hooks_url":"https://api.github.com/repos/stratosphere/stratosphere/hooks","issue_events_url":"https://api.github.com/repos/stratosphere/stratosphere/issues/events{/number}","events_url":"https://api.github.com/repos/stratosphere/stratosphere/events","assignees_url":"https://api.github.com/repos/stratosphere/stratosphere/assignees{/user}","branches_url":"https://api.github.com/repos/stratosphere/stratosphere/branches{/branch}","tags_url":"https://api.github.com/repos/stratosphere/stratosphere/tags","blobs_url":"https://api.github.com/repos/stratosphere/stratosphere/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/stratosphere/stratosphere/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/stratosphere/stratosphere/git/refs{/sha}","trees_url":"https://api.github.com/repos/stratosphere/stratosphere/git/trees{/sha}","statuses_url":"https://api.github.com/repos/stratosphere/stratosphere/statuses/{sha}","languages_url":"https://api.github.com/repos/stratosphere/stratosphere/languages","stargazers_url":"https://api.github.com/repos/stratosphere/stratosphere/stargazers","contributors_url":"https://api.github.com/repos/stratosphere/stratosphere/contributors","subscribers_url":"https://api.github.com/repos/stratosphere/stratosphere/subscribers","subscription_url":"https://api.github.com/repos/stratosphere/stratosphere/subscription","commits_url":"https://api.github.com/repos/stratosphere/stratosphere/commits{/sha}","git_commits_url":"https://api.github.com/repos/stratosphere/stratosphere/git/commits{/sha}","comments_url":"https://api.github.com/repos/stratosphere/stratosphere/comments{/number}","issue_comment_url":"https://api.github.com/repos/stratosphere/stratosphere/issues/comments/{number}","contents_url":"https://api.github.com/repos/stratosphere/stratosphere/contents/{+path}","compare_url":"https://api.github.com/repos/stratosphere/stratosphere/compare/{base}...{head}","merges_url":"https://api.github.com/repos/stratosphere/stratosphere/merges","archive_url":"https://api.github.com/repos/stratosphere/stratosphere/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/stratosphere/stratosphere/downloads","issues_url":"https://api.github.com/repos/stratosphere/stratosphere/issues{/number}","pulls_url":"https://api.github.com/repos/stratosphere/stratosphere/pulls{/number}","milestones_url":"https://api.github.com/repos/stratosphere/stratosphere/milestones{/number}","notifications_url":"https://api.github.com/repos/stratosphere/stratosphere/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/stratosphere/stratosphere/labels{/name}","releases_url":"https://api.github.com/repos/stratosphere/stratosphere/releases{/id}","created_at":"2013-04-17T14:25:24Z","updated_at":"2014-03-25T20:05:26Z","pushed_at":"2014-03-25T20:05:26Z","git_url":"git://github.com/stratosphere/stratosphere.git","ssh_url":"git@github.com:stratosphere/stratosphere.git","clone_url":"https://github.com/stratosphere/stratosphere.git","svn_url":"https://github.com/stratosphere/stratosphere","homepage":"http://stratosphere.eu","size":100970,"stargazers_count":122,"watchers_count":122,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"forks_count":82,"mirror_url":null,"open_issues_count":151,"forks":82,"open_issues":151,"watchers":122,"default_branch":"master","master_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/stratosphere/stratosphere/pulls/511"},"html":{"href":"https://github.com/stratosphere/stratosphere/pull/511"},"issue":{"href":"https://api.github.com/repos/stratosphere/stratosphere/issues/511"},"comments":{"href":"https://api.github.com/repos/stratosphere/stratosphere/issues/511/comments"},"review_comments":{"href":"https://api.github.com/repos/stratosphere/stratosphere/pulls/511/comments"},"review_comment":{"href":"https://api.github.com/repos/stratosphere/stratosphere/pulls/comments/{number}"},"commits":{"href":"https://api.github.com/repos/stratosphere/stratosphere/pulls/511/commits"},"statuses":{"href":"https://api.github.com/repos/stratosphere/stratosphere/statuses/cd04a3361d95ad9f359c0846929de0dcb3fcbbc3"}},"merged":false,"mergeable":false,"mergeable_state":"dirty","merged_by":null,"comments":1,"review_comments":0,"commits":1,"additions":7706,"deletions":15019,"changed_files":295}}