GitHub PullRequestEvent at 2014-11-18 17:41:21
Repository: caffe
Actor: mtamburrano
Payload: {"action":"opened","number":1448,"pull_request":{"url":"https://api.github.com/repos/BVLC/caffe/pulls/1448","id":24629571,"html_url":"https://github.com/BVLC/caffe/pull/1448","diff_url":"https://github.com/BVLC/caffe/pull/1448.diff","patch_url":"https://github.com/BVLC/caffe/pull/1448.patch","issue_url":"https://api.github.com/repos/BVLC/caffe/issues/1448","number":1448,"state":"open","locked":false,"title":"Conditional layer","user":{"login":"mtamburrano","id":1943264,"avatar_url":"https://avatars.githubusercontent.com/u/1943264?v=3","gravatar_id":"","url":"https://api.github.com/users/mtamburrano","html_url":"https://github.com/mtamburrano","followers_url":"https://api.github.com/users/mtamburrano/followers","following_url":"https://api.github.com/users/mtamburrano/following{/other_user}","gists_url":"https://api.github.com/users/mtamburrano/gists{/gist_id}","starred_url":"https://api.github.com/users/mtamburrano/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mtamburrano/subscriptions","organizations_url":"https://api.github.com/users/mtamburrano/orgs","repos_url":"https://api.github.com/users/mtamburrano/repos","events_url":"https://api.github.com/users/mtamburrano/events{/privacy}","received_events_url":"https://api.github.com/users/mtamburrano/received_events","type":"User","site_admin":false},"body":"This Layer, developed by me and @bhack , adds the possibility to select a subset of the batch whose items\r\npass a conditional test. It includes test cases and doxygen in common_layers.hpp.\r\n\r\nThe layer consists of:\r\n\r\n3 bottom blobs:\r\n\r\n - bottom\\[0\\]: this is the \"bottom_IF\". For each item, an argmax is computed\r\n    and the index of the max value is compared with the layer parameter\r\n    *conditional_index*. If *index_argmax* == *conditional_index*, the index\r\n    of the item in the blob is stored in a vector, let's call this vector\r\n    *indices_to_forward*.\r\n - bottom\\[1\\]: this is the \"bottom_THEN\". Only the items whose indices are contained\r\n    by the *indices_to_forward* vector will be forwarded.\r\n - bottom\\[2\\]: this is the bottom which contains the labels. If the layer parameter\r\n    *output_type* is equal to FILTERED_LABELS, then only the labels whose indices \r\n    are contained by the *indices_to_forward* vector will be forwarded,\r\n    otherwise this bottom is ignored.\r\n\r\n2 top blobs:\r\n\r\n - top[0]: If the layer parameter *output_type* == *FILTERED_LABELS*, then this blob\r\n    will contains the filtered labels as explained above. If *output_type* == *FILTERED_INDICES*\r\n    this blob will contains the indices of items that passed the conditional tests\r\n    (the same indices contained in the *indices_to_forward* vector), especially\r\n    useful for predicting.\r\n - top\\[1\\]: This blob will contains the filtered items from bottom[1]\r\n\r\nParams:\r\n\r\n - **int** *conditional_index*:\r\n    The index which will be compared with the argmax index of each item in the bottom[0]\r\n - **int** *output_type*:\r\n    two different output types for top[0]. *FILTERED_LABELS*, useful for training/testing,\r\n    and *FILTERED_INDICES*, useful for predicting.\r\n    \r\nExample:\r\nLet's say we want a net that distinguishes between \"cat\" and \"NOT-cat\", and tries to recognize\r\nthe breed.\r\nIf the net recognizes an input as \"NOT-cat\", is useless to perform the remaining forward passes that try\r\nto discriminate among the cat breeds.\r\nSo we could have something like this:\r\n\r\n                      INPUT_DATA\r\n                          |\r\n                      CONV_LAYER1\r\n                          |\r\n                        POOL1\r\n                        /    \\\r\n                      /        \\\r\n              INNER_PRODUCT1    CONV_LAYER2\r\n              /                           |\r\n          SOFTMAX1                      POOL2\r\n      //outputs 0 or 1 if                 |\r\n      // is \"NON-cat\" or \"cat\"        INNER_PRODUCT2\r\n                                          |\r\n                                        SOFTMAX2\r\n                                  //outputs the cat's breed \r\n                                      \r\nIn this net, no matter what SOFTMAX1 produces, all the data input will be always forwarded\r\nto the other branch of the net. So with conditional layer we could build the following net:\r\n\r\n           INPUT_DATA\r\n              |\r\n          CONV_LAYER1\r\n              |\r\n            POOL1\r\n              |\r\n          INNER_PRODUCT1\r\n              |\r\n            SOFTMAX1 \r\n    //outputs 0 or 1 if is \"NON-cat\" or \"cat\"\r\n              |\r\n          CONDITIONAL:\r\n          {\r\n            bottom[0]: SOFTMAX1\r\n            bottom[1]: POOL1\r\n            bottom[2]: INPUT_DATA_labels\r\n            top[0]: FILTERED_INPUT_DATA_labels\r\n            top[1]: CONDITIONAL_POOL1\r\n            conditional_parameters\r\n            {\r\n              conditional_index: 1\r\n              output_type: FILTERED_LABELS\r\n            }\r\n          }\r\n              |\r\n          CONV_LAYER2\r\n              |\r\n          INNER_PRODUCT2\r\n              |\r\n            POOL2\r\n              |\r\n            SOFTMAX2 //outputs the cat's breed\r\n\r\nNow, when the net reaches the conditional layer, the output of SOFTMAX1 is evaluated\r\nto find those items that have SOFTMAX\\[1\\] >= SOFTMAX\\[0\\] (because *conditional_index* == 1, if\r\nSOFTMAX\\[1\\] > SOFTMAX\\[0\\] we can presume that the item is a cat). Then we store the indices\r\nof these items and forward from POOL1 to CONV_LAYER2 only the items with those indices \r\n(or else, the cats!).\r\n      \r\nIn the train phase this allows to train only a part of the input data in a specific branch\r\nof the net, maintaining the possibility to have shared layers. In the predict phase a conditional layer\r\ncould save a lot of time: in the previous example probably the most computional expensive part\r\nof the net would be the one that discriminates among the breeds, so all the forward passes for\r\nthe NON-cats items are spared.\r\nFor predicting is useful to use *output_type* with *FILTERED_INDICES*, so you can distinguish which of\r\nthe input data are been forwarded until the end of the net and which stopped before.\r\n\r\nIn addition, the conditional_layer can be used as logic ports. For example:\r\n\r\n          \r\n           GENERIC_LAYER1\r\n          /             \\\r\n    CONDITIONAL_L1    CONDITIONAL_L2\r\n          \\             /\r\n           GENERIC_LAYER2\r\n\r\nsimulates an OR, on the contrary:\r\n\r\n       GENERIC_LAYER1\r\n              |\r\n        CONDITIONAL_L1\r\n              |\r\n        CONDITIONAL_L2\r\n              |\r\n       GENERIC_LAYER2\r\n       \r\nsimulates an AND.\r\n\r\n#####NOTES:\r\nIf used for predicting, the conditional layer works fine, instead if used in the train/test phase, actually can't works in DEV. This is because each LOSS Layer has the following check:\r\n\r\n    if (propagate_down[1]) {\r\n    LOG(FATAL) << this->type_name()\r\n               << \" Layer cannot backpropagate to label inputs.\";\r\n    }\r\ninto the backward function. The problem is that my layer needs to backpropagate Top[1] but not Top\\[0\\] (that contains labels) but actually seems this is not possible because a layer needs or not needs backpropagation on ALL its blobs. To test my layer I commented that check in the LOSS layer and everything seems to work fine, the net converges normally. So how can I fix this? It is possible to remove that check in the LOSS layers? I presume it is there just to be sure someone has not made up the net wrong, attaching a non-label blob to a loss layer, right?","created_at":"2014-11-18T17:41:21Z","updated_at":"2014-11-18T17:41:21Z","closed_at":null,"merged_at":null,"merge_commit_sha":null,"assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/BVLC/caffe/pulls/1448/commits","review_comments_url":"https://api.github.com/repos/BVLC/caffe/pulls/1448/comments","review_comment_url":"https://api.github.com/repos/BVLC/caffe/pulls/comments/{number}","comments_url":"https://api.github.com/repos/BVLC/caffe/issues/1448/comments","statuses_url":"https://api.github.com/repos/BVLC/caffe/statuses/d8c4482a6abb6bfa95f3cd156d0544ae5b05a0e5","head":{"label":"mtamburrano:conditional_layer","ref":"conditional_layer","sha":"d8c4482a6abb6bfa95f3cd156d0544ae5b05a0e5","user":{"login":"mtamburrano","id":1943264,"avatar_url":"https://avatars.githubusercontent.com/u/1943264?v=3","gravatar_id":"","url":"https://api.github.com/users/mtamburrano","html_url":"https://github.com/mtamburrano","followers_url":"https://api.github.com/users/mtamburrano/followers","following_url":"https://api.github.com/users/mtamburrano/following{/other_user}","gists_url":"https://api.github.com/users/mtamburrano/gists{/gist_id}","starred_url":"https://api.github.com/users/mtamburrano/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mtamburrano/subscriptions","organizations_url":"https://api.github.com/users/mtamburrano/orgs","repos_url":"https://api.github.com/users/mtamburrano/repos","events_url":"https://api.github.com/users/mtamburrano/events{/privacy}","received_events_url":"https://api.github.com/users/mtamburrano/received_events","type":"User","site_admin":false},"repo":{"id":26318920,"name":"caffe","full_name":"mtamburrano/caffe","owner":{"login":"mtamburrano","id":1943264,"avatar_url":"https://avatars.githubusercontent.com/u/1943264?v=3","gravatar_id":"","url":"https://api.github.com/users/mtamburrano","html_url":"https://github.com/mtamburrano","followers_url":"https://api.github.com/users/mtamburrano/followers","following_url":"https://api.github.com/users/mtamburrano/following{/other_user}","gists_url":"https://api.github.com/users/mtamburrano/gists{/gist_id}","starred_url":"https://api.github.com/users/mtamburrano/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mtamburrano/subscriptions","organizations_url":"https://api.github.com/users/mtamburrano/orgs","repos_url":"https://api.github.com/users/mtamburrano/repos","events_url":"https://api.github.com/users/mtamburrano/events{/privacy}","received_events_url":"https://api.github.com/users/mtamburrano/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/mtamburrano/caffe","description":"Caffe: a fast framework for deep learning. For the most recent version checkout the dev branch. For the latest stable release checkout the master branch.","fork":true,"url":"https://api.github.com/repos/mtamburrano/caffe","forks_url":"https://api.github.com/repos/mtamburrano/caffe/forks","keys_url":"https://api.github.com/repos/mtamburrano/caffe/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mtamburrano/caffe/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mtamburrano/caffe/teams","hooks_url":"https://api.github.com/repos/mtamburrano/caffe/hooks","issue_events_url":"https://api.github.com/repos/mtamburrano/caffe/issues/events{/number}","events_url":"https://api.github.com/repos/mtamburrano/caffe/events","assignees_url":"https://api.github.com/repos/mtamburrano/caffe/assignees{/user}","branches_url":"https://api.github.com/repos/mtamburrano/caffe/branches{/branch}","tags_url":"https://api.github.com/repos/mtamburrano/caffe/tags","blobs_url":"https://api.github.com/repos/mtamburrano/caffe/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mtamburrano/caffe/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mtamburrano/caffe/git/refs{/sha}","trees_url":"https://api.github.com/repos/mtamburrano/caffe/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mtamburrano/caffe/statuses/{sha}","languages_url":"https://api.github.com/repos/mtamburrano/caffe/languages","stargazers_url":"https://api.github.com/repos/mtamburrano/caffe/stargazers","contributors_url":"https://api.github.com/repos/mtamburrano/caffe/contributors","subscribers_url":"https://api.github.com/repos/mtamburrano/caffe/subscribers","subscription_url":"https://api.github.com/repos/mtamburrano/caffe/subscription","commits_url":"https://api.github.com/repos/mtamburrano/caffe/commits{/sha}","git_commits_url":"https://api.github.com/repos/mtamburrano/caffe/git/commits{/sha}","comments_url":"https://api.github.com/repos/mtamburrano/caffe/comments{/number}","issue_comment_url":"https://api.github.com/repos/mtamburrano/caffe/issues/comments/{number}","contents_url":"https://api.github.com/repos/mtamburrano/caffe/contents/{+path}","compare_url":"https://api.github.com/repos/mtamburrano/caffe/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mtamburrano/caffe/merges","archive_url":"https://api.github.com/repos/mtamburrano/caffe/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mtamburrano/caffe/downloads","issues_url":"https://api.github.com/repos/mtamburrano/caffe/issues{/number}","pulls_url":"https://api.github.com/repos/mtamburrano/caffe/pulls{/number}","milestones_url":"https://api.github.com/repos/mtamburrano/caffe/milestones{/number}","notifications_url":"https://api.github.com/repos/mtamburrano/caffe/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mtamburrano/caffe/labels{/name}","releases_url":"https://api.github.com/repos/mtamburrano/caffe/releases{/id}","created_at":"2014-11-07T12:30:42Z","updated_at":"2014-11-07T10:53:42Z","pushed_at":"2014-11-18T16:28:48Z","git_url":"git://github.com/mtamburrano/caffe.git","ssh_url":"git@github.com:mtamburrano/caffe.git","clone_url":"https://github.com/mtamburrano/caffe.git","svn_url":"https://github.com/mtamburrano/caffe","homepage":"http://caffe.berkeleyvision.org/","size":30718,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"BVLC:dev","ref":"dev","sha":"eca019aca7efce5293bb3a821b4b20cb401e384e","user":{"login":"BVLC","id":6182163,"avatar_url":"https://avatars.githubusercontent.com/u/6182163?v=3","gravatar_id":"","url":"https://api.github.com/users/BVLC","html_url":"https://github.com/BVLC","followers_url":"https://api.github.com/users/BVLC/followers","following_url":"https://api.github.com/users/BVLC/following{/other_user}","gists_url":"https://api.github.com/users/BVLC/gists{/gist_id}","starred_url":"https://api.github.com/users/BVLC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/BVLC/subscriptions","organizations_url":"https://api.github.com/users/BVLC/orgs","repos_url":"https://api.github.com/users/BVLC/repos","events_url":"https://api.github.com/users/BVLC/events{/privacy}","received_events_url":"https://api.github.com/users/BVLC/received_events","type":"Organization","site_admin":false},"repo":{"id":12791642,"name":"caffe","full_name":"BVLC/caffe","owner":{"login":"BVLC","id":6182163,"avatar_url":"https://avatars.githubusercontent.com/u/6182163?v=3","gravatar_id":"","url":"https://api.github.com/users/BVLC","html_url":"https://github.com/BVLC","followers_url":"https://api.github.com/users/BVLC/followers","following_url":"https://api.github.com/users/BVLC/following{/other_user}","gists_url":"https://api.github.com/users/BVLC/gists{/gist_id}","starred_url":"https://api.github.com/users/BVLC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/BVLC/subscriptions","organizations_url":"https://api.github.com/users/BVLC/orgs","repos_url":"https://api.github.com/users/BVLC/repos","events_url":"https://api.github.com/users/BVLC/events{/privacy}","received_events_url":"https://api.github.com/users/BVLC/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/BVLC/caffe","description":"Caffe: a fast framework for deep learning. For the most recent version checkout the dev branch. For the latest stable release checkout the master branch.","fork":false,"url":"https://api.github.com/repos/BVLC/caffe","forks_url":"https://api.github.com/repos/BVLC/caffe/forks","keys_url":"https://api.github.com/repos/BVLC/caffe/keys{/key_id}","collaborators_url":"https://api.github.com/repos/BVLC/caffe/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/BVLC/caffe/teams","hooks_url":"https://api.github.com/repos/BVLC/caffe/hooks","issue_events_url":"https://api.github.com/repos/BVLC/caffe/issues/events{/number}","events_url":"https://api.github.com/repos/BVLC/caffe/events","assignees_url":"https://api.github.com/repos/BVLC/caffe/assignees{/user}","branches_url":"https://api.github.com/repos/BVLC/caffe/branches{/branch}","tags_url":"https://api.github.com/repos/BVLC/caffe/tags","blobs_url":"https://api.github.com/repos/BVLC/caffe/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/BVLC/caffe/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/BVLC/caffe/git/refs{/sha}","trees_url":"https://api.github.com/repos/BVLC/caffe/git/trees{/sha}","statuses_url":"https://api.github.com/repos/BVLC/caffe/statuses/{sha}","languages_url":"https://api.github.com/repos/BVLC/caffe/languages","stargazers_url":"https://api.github.com/repos/BVLC/caffe/stargazers","contributors_url":"https://api.github.com/repos/BVLC/caffe/contributors","subscribers_url":"https://api.github.com/repos/BVLC/caffe/subscribers","subscription_url":"https://api.github.com/repos/BVLC/caffe/subscription","commits_url":"https://api.github.com/repos/BVLC/caffe/commits{/sha}","git_commits_url":"https://api.github.com/repos/BVLC/caffe/git/commits{/sha}","comments_url":"https://api.github.com/repos/BVLC/caffe/comments{/number}","issue_comment_url":"https://api.github.com/repos/BVLC/caffe/issues/comments/{number}","contents_url":"https://api.github.com/repos/BVLC/caffe/contents/{+path}","compare_url":"https://api.github.com/repos/BVLC/caffe/compare/{base}...{head}","merges_url":"https://api.github.com/repos/BVLC/caffe/merges","archive_url":"https://api.github.com/repos/BVLC/caffe/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/BVLC/caffe/downloads","issues_url":"https://api.github.com/repos/BVLC/caffe/issues{/number}","pulls_url":"https://api.github.com/repos/BVLC/caffe/pulls{/number}","milestones_url":"https://api.github.com/repos/BVLC/caffe/milestones{/number}","notifications_url":"https://api.github.com/repos/BVLC/caffe/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/BVLC/caffe/labels{/name}","releases_url":"https://api.github.com/repos/BVLC/caffe/releases{/id}","created_at":"2013-09-12T18:39:48Z","updated_at":"2014-11-18T16:50:41Z","pushed_at":"2014-11-14T06:14:38Z","git_url":"git://github.com/BVLC/caffe.git","ssh_url":"git@github.com:BVLC/caffe.git","clone_url":"https://github.com/BVLC/caffe.git","svn_url":"https://github.com/BVLC/caffe","homepage":"http://caffe.berkeleyvision.org/","size":57747,"stargazers_count":1512,"watchers_count":1512,"language":"C++","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":814,"mirror_url":null,"open_issues_count":219,"forks":814,"open_issues":219,"watchers":1512,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/BVLC/caffe/pulls/1448"},"html":{"href":"https://github.com/BVLC/caffe/pull/1448"},"issue":{"href":"https://api.github.com/repos/BVLC/caffe/issues/1448"},"comments":{"href":"https://api.github.com/repos/BVLC/caffe/issues/1448/comments"},"review_comments":{"href":"https://api.github.com/repos/BVLC/caffe/pulls/1448/comments"},"review_comment":{"href":"https://api.github.com/repos/BVLC/caffe/pulls/comments/{number}"},"commits":{"href":"https://api.github.com/repos/BVLC/caffe/pulls/1448/commits"},"statuses":{"href":"https://api.github.com/repos/BVLC/caffe/statuses/d8c4482a6abb6bfa95f3cd156d0544ae5b05a0e5"}},"merged":false,"mergeable":null,"mergeable_state":"unknown","merged_by":null,"comments":0,"review_comments":0,"commits":6,"additions":533,"deletions":2,"changed_files":5}}