Exhibit 10.36

EXECUTION COPY

FIFTH AMENDMENT AGREEMENT

To Pledge and Security Agreement and Irrevocable Proxy

Dated as of June 30, 2009

by and among

RFC ASSET
HOLDINGS II, LLC,

PASSIVE ASSET TRANSACTIONS, LLC

AND CERTAIN

AFFILIATES THEREOF FROM

TIME TO TIME PARTY HERETO,

as Grantors,

and

GMAC LLC,

as Lender Agent



| |  |  |  |                                                           
| |  |  |  | ----------------------------------------------------------
| |  |  |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

This FIFTH AMENDMENT AGREEMENT (this Agreement), dated as of June 30, 2009 (the
Amendment Effective Date), is by and among RFC Asset Holdings II, LLC, a Delaware limited liability company (RAHI), and Passive Asset Transactions, LLC, a Delaware limited liability company
(PATI; and together with RAHI, each a Borrower and collectively, the Borrowers); Residential Capital, LLC, a Delaware limited liability company (ResCap), Residential Funding
Company, LLC, a Delaware limited liability company (RFC), and GMAC Mortgage, LLC, a Delaware limited liability company (GMAC Mortgage; and together with ResCap and RFC, each herein a
Guarantor and collectively, the Guarantors), and the other parties hereto as Grantors (each, together with each Borrower and each Guarantor, a Grantor and collectively, the
Grantors); and GMAC LLC, a Delaware limited liability company, as Lender Agent for the Lender Parties.

Reference
is hereby made to the Pledge and Security Agreement and Irrevocable Proxy dated as of November 20, 2008 among the Grantors and the Lender Agent (as amended and modified through the date hereof, the Security Agreement).

RECITALS

1. Each of the
parties hereto is a party to the Security Agreement.

2. The parties hereto desire to make certain amendments to the Security Agreement.

3. Each of the parties hereto, by its signature hereto, hereby acknowledges, consents and agrees to the changes set forth herein.

4. In consideration of the promises and mutual agreements herein contained and for good and valuable consideration, the receipt and
sufficiency of which is hereby acknowledged, the parties hereto agree as follows:

ARTICLE I

DEFINED TERMS

SECTION 1.1 Capitalized terms
used herein and not otherwise defined shall have the meaning set forth in the Security Agreement.

ARTICLE II

AMENDMENTS TO THE SECURITY AGREEMENT

SECTION
2.1 Amendments to Section 1.

(a) The following definitions in Section 1 to the Security Agreement
are hereby amended and restated in their entirety to read as follows:

First Savings Warehouse Note means that Promissory Note, dated
as of April 1, 2005 and that Promissory Note dated as of July 1, 2009 and issued by First Savings Mortgage Corporation in favor of RFC, as the same may be amended, supplemented, restated or otherwise modified from time to time, and
including any notes given in substitution or replacement therefor.



| |  |  |  |                                                           
| |  |  |  | ----------------------------------------------------------
| |  |  |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

(b) The following definitions are hereby added to Section 1 to the Security
Agreement:

MSR Documents means (a) the MSR Loan Agreement, (b) the Notes,  the Guarantee, the
Account Control Agreements, and the Servicing Contracts (each as defined in the MSR Loan Agreement) and (c) all notices, certificates, financing statements and other documents to be executed and delivered by RFC,
GMAC Mortgage or ResCap in connection with the transactions contemplated by the MSR Loan Agreement.

November Documents means
(a) the Loan Agreement, the Notes, this Agreement, the Account Control Agreements, the GSAP Indenture Transaction Documents and the Flume No. 8 Security Documents, (b) after the Custodial Transfer Date, the Master Custody Agreement,
(c) any document designated as a Security Document in any Collateral Addition Designation Notice, (d) all of the security agreements, pledges, collateral assignments, mortgages, deeds of trust, trust deeds or other instruments
evidencing or creating or purporting to create any security interests in favor of the Lender Agent for its benefit and for the benefit of the Lender Parties, (e) the Contribution Agreements and (f) all notices, certificates, financing
statements, agreements and other documents to be executed and delivered by RAHI, PATI, ResCap, RFC or GMAC Mortgage pursuant to the foregoing or otherwise in connection with the Loan Agreement or the extension of financing by the Lenders
contemplated thereunder.

Warehouse Loans means, as the context may require, loans made by RFC pursuant to a Warehouse Agreement to
the borrower under such agreement and/or any mortgage loans or other loans or assets purchased by RFC pursuant to the First Savings Warehouse Agreement.

(c) The definitions of Permitted Liens, Secured Parties and Credit Agent are hereby deleted in their entirety from Section 1 to the Security Agreement.

SECTION 2.2 Amendments to Schedule IV.

(a) Exhibit A to Schedule IV of the Fourth Security Agreement is hereby amended by inserting the following:



|                   |  |                     |  |               |  |                   
|------------------ |  | ------------------- |  | ------------- |  | ------------------
|GMAC Mortgage, LLC |  | Wachovia Bank, N.A. |  | 2000049234945 |  | GMAC Mortgage, LLC



| |  |   |  |                                                           
| |  | - |  | ----------------------------------------------------------
| |  | 2 |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

(b) Exhibit C to Schedule IV of the Security Agreement is hereby amended by amending and restating the
third Pledged Note listed therein as follows:



|                                   |  |                                                                                                                                                                                                                                                                                                                        |  |                                 
|---------------------------------- |  | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |  | --------------------------------
|First Savings Mortgage Corporation |  | All First Savings Warehouse Notes, including, without limitation, that certain Promissory Note, dated as of April 1, 2005 and issued by First Savings Mortgage Corporation in favor of
RFC and that certain Promissory Note, dated as of July 1, 2009 and issued by First Savings Mortgage Corporation in favor of RFC |  | Residential Funding Company, LLC

(c) Exhibit D to Schedule IV of the Security Agreement is hereby amended by inserting the
following:



|                               |  |                           |  |      |  |                                 |  |                                                                                                                                                                            
|------------------------------ |  | ------------------------- |  | ---- |  | ------------------------------- |  | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|PATI Real Estate Holdings, LLC |  | Limited Liability Company |  | 100% |  | Passive Asset Transactions, LLC |  | 100% of Passive Asset Transactions, LLCs equity interest in PATI Real Estate Holdings, LLC, constituting 100% of all the member interest in PATI Real Estate Holdings, LLC
|                               |  |                           |  |     
|RAHI Real Estate Holdings, LLC |  | Limited Liability Company |  | 100% |  | RFC Asset Holdings II, LLC      |  | 100% of RFC Asset Holdings II, LLCs equity interest in RAHI Real Estate Holdings, LLC, constituting 100% of all the member interest in RAHI Real Estate Holdings,
LLC     



| |  |   |  |                                                           
| |  | - |  | ----------------------------------------------------------
| |  | 3 |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

ARTICLE III

CONDITIONS TO EFFECTIVENESS

SECTION 3.1 Amendment Effective Date. This Agreement and the provisions
contained herein shall become effective as of the Amendment Effective Date.

SECTION 3.2 Opinions and Other Deliverables. The
Obligors covenant and agree to deliver or cause to be delivered (a) opinions of counsel to the Obligors with respect to the transactions contemplated hereby, which opinions shall be in form and substance satisfactory to the Lender Agent, on or
before July 8, 2009 and (b) such other documents, including but not limited to the Obligors board resolutions approving this Agreement, as the Lender Agent may reasonably request, which documents will be in form and substance
satisfactory to the Lender Agent, on or before July 15, 2009. The Obligors acknowledge and agree that the Lender Agent may, upon the failure to deliver any of the items set forth in clauses (a) and (b) above in the timeframes set
forth therein, declare an Event of Default.

ARTICLE IV

ACKNOWLEDGEMENTS, CONSENTS, NOTICE, CONFIRMATION AND

REPRESENTATIONS AND WARRANTIES

SECTION 4.1 Notice. Each party hereto hereby acknowledges timely notice of the execution of this Agreement and of the transactions and amendments
contemplated hereby. Each party hereto hereby waives any notice requirement contained in the Security Agreement or the Facility Documents with respect to the execution of this Agreement.

SECTION 4.2 Confirmation of the Security Agreement. The Grantors each hereby acknowledge and agree that the Security Agreement and each other
Facility Document (each as amended as of the date hereof) are each ratified and confirmed in all respects and shall remain in full force and effect in accordance with their respective terms. Without limiting the foregoing, each Grantor reaffirms its
grant of a security interest in all the Collateral pledged by it, and agrees that such security interest secures all Obligations as defined in the Loan Agreement. As of the Amendment Effective Date, each reference in the Security Agreement to
this Agreement or in any other Facility Document to the Security Agreement shall mean the Security Agreement as amended by this Agreement, and as hereinafter amended or restated.

SECTION 4.3 Representations and Warranties. By its signature hereto, each Grantor hereby represents and warrants that, before and after giving
effect to this Agreement, as follows:

(a) Its representations and warranties set forth in the Facility Documents (each as
amended as of the date hereof) are true and correct as if made on the date hereof, except to the extent they expressly relate to an earlier date; and

(b) No Default has occurred and is continuing.



| |  |   |  |                                                           
| |  | - |  | ----------------------------------------------------------
| |  | 4 |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

ARTICLE V

MISCELLANEOUS

SECTION 5.1 GOVERNING LAW. THIS AGREEMENT SHALL BE GOVERNED BY AND CONSTRUED IN
ACCORDANCE WITH THE LAWS OF THE STATE OF NEW YORK WITHOUT REGARD TO CONFLICTS OF LAWS PRINCIPLES (BUT WITH REFERENCE TO SECTION 5-1401 OF THE NEW YORK GENERAL OBLIGATIONS LAW, WHICH BY ITS TERMS APPLIES TO THIS AGREEMENT).

SECTION 5.2 Execution in Counterparts. This Agreement may be executed in any number of counterparts and by different parties hereto in separate
counterparts, each of which when so executed shall be deemed to be an original (whether such counterpart is originally executed or an electronic copy of an original and each party hereto expressly waives its rights to receive originally executed
documents) and all of which when taken together shall constitute one and the same agreement.

SECTION 5.3 WAIVER OF JURY TRIAL. EACH
OF THE PARTIES HEREBY IRREVOCABLY WAIVES ANY AND ALL RIGHT TO A TRIAL BY JURY WITH RESPECT TO ANY LEGAL PROCEEDING ARISING OUT OF OR RELATING TO THIS AGREEMENT.

SECTION 5.4 Entire Agreement. This Agreement, the Security Agreement and the other Facility Documents embody the entire agreement and understanding of the parties hereto and supersede any and all prior
agreements, arrangements and understanding relating to the matters provided for herein.

SECTION 5.5 Captions. The various captions
in this Agreement are included for convenience only and shall not affect the meaning or interpretation of any provision of this Agreement.

SECTION 5.6 Severability. If any provision of this Agreement, or the application thereof to any party or any circumstance, is held to be unenforceable, invalid or illegal (in whole or in part) for any reason (in any jurisdiction),
the remaining terms of this Agreement, modified by the deletion of the unenforceable invalid or illegal portion (in any relevant jurisdiction), will continue in full force and effect, and such unenforceability, invalidity or illegality will not
otherwise affect the enforceability, validity or legality of the remaining terms of this Agreement so long as this Agreement, as so modified, continues to express, without material change, the original intentions of the parties as to the subject
matter hereof and the deletion of such portion of this Agreement will not substantially impair the respective expectations of the parties or the practical realization of the benefits that would otherwise be conferred upon the parties.



| |  |   |  |                                                           
| |  | - |  | ----------------------------------------------------------
| |  | 5 |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

SECTION 5.7 SUBMISSION TO JURISDICTION. EACH PARTY HERETO HEREBY IRREVOCABLY SUBMITS TO THE
NONEXCLUSIVE JURISDICTION OF THE COURTS OF THE STATE OF NEW YORK LOCATED IN THE CITY OF MANHATTAN OR IN THE UNITED STATES DISTRICT COURT FOR THE SOUTHERN DISTRICT OF NEW YORK FOR PURPOSES OF ALL LEGAL PROCEEDINGS ARISING OUT OF OR RELATING TO THIS
AGREEMENT OR THE TRANSACTIONS CONTEMPLATED HEREBY AND IRREVOCABLY AGREES TO BE BOUND BY ANY JUDGMENT RENDERED THEREBY IN CONNECTION WITH SUCH LITIGATION. EACH PARTY HERETO IRREVOCABLY WAIVES, TO THE FULLEST EXTENT PERMITTED BY LAW, ANY OBJECTION
WHICH IT MAY NOW OR HEREAFTER HAVE TO THE LAYING OF THE VENUE OF ANY SUCH PROCEEDING BROUGHT IN SUCH A COURT AND ANY CLAIM THAT ANY SUCH PROCEEDING BROUGHT IN SUCH A COURT HAS BEEN BROUGHT IN AN INCONVENIENT FORUM. EACH PARTY HERETO HEREBY CONSENTS
TO PROCESS BEING SERVED IN ANY SUIT, ACTION OR PROCEEDING WITH RESPECT TO THIS AGREEMENT, OR ANY DOCUMENT DELIVERED PURSUANT HERETO BY THE MAILING OF A COPY THEREOF BY REGISTERED OR CERTIFIED MAIL, POSTAGE PREPAID, RETURN RECEIPT REQUESTED, TO ITS
RESPECTIVE ADDRESS SPECIFIED AT THE TIME FOR NOTICES UNDER THIS AGREEMENT OR TO ANY OTHER ADDRESS OF WHICH IT SHALL HAVE GIVEN WRITTEN OR ELECTRONIC NOTICE TO THE OTHER PARTIES. THE FOREGOING SHALL NOT LIMIT THE ABILITY OF ANY PARTY HERETO TO BRING
SUIT IN THE COURTS OF ANY JURISDICTION.

[REMAINDER OF PAGE INTENTIONALLY LEFT BLANK]



| |  |   |  |                                                           
| |  | - |  | ----------------------------------------------------------
| |  | 6 |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

IN WITNESS WHEREOF, the parties have caused this Agreement to be executed by their respective officers
thereunto duly authorized, as of the date first above written.



RFC ASSET HOLDINGS II, LLC, as
GrantorBy:

/s/ Melissa White


Name:

Melissa WhiteTitle:

Assistant Treasurer




| |  |     |  |                                                           
| |  | --- |  | ----------------------------------------------------------
| |  | S-1 |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

PASSIVE ASSET TRANSACTIONS, LLC, as GrantorBy:

/s/ Melissa White


Name:

Melissa WhiteTitle:

Assistant Treasurer




| |  |     |  |                                                           
| |  | --- |  | ----------------------------------------------------------
| |  | S-2 |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

RESIDENTIAL CAPITAL, LLC, as
GrantorBy:

/s/ John M. Peterson


Name:

John M. PetersonTitle:

Assistant Treasurer




| |  |     |  |                                                           
| |  | --- |  | ----------------------------------------------------------
| |  | S-3 |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

RESIDENTIAL FUNDING COMPANY, LLC, as GrantorBy:

/s/ Melissa White


Name:

Melissa WhiteTitle:

Assistant Treasurer




| |  |     |  |                                                           
| |  | --- |  | ----------------------------------------------------------
| |  | S-4 |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

GMAC MORTGAGE, LLC, as
GrantorBy:

/s/ Melissa White


Name:

Melissa WhiteTitle:

Assistant Treasurer




| |  |     |  |                                                           
| |  | --- |  | ----------------------------------------------------------
| |  | S-5 |  | Fifth Amendment Agreement to Pledge and Security Agreement



---

GMAC LLC, as Lender
AgentBy:

/s/ David C. Walker


Name:

David C. WalkerTitle:

Business Unit Treasury Executive




| |  |     |  |                                                           
| |  | --- |  | ----------------------------------------------------------
| |  | S-6 |  | Fifth Amendment Agreement to Pledge and Security Agreement