<PAGE>
 
                                                                    Exhibit 10.1

                             PAMRAPO BANCORP, INC.
                          CHANGE IN CONTROL AGREEMENT


     This AGREEMENT is made effective as of January 1, 1997, by and between
Pamrapo Bancorp, Inc. (the "Holding Company"), a corporation organized under the
laws of the State of Delaware, with its office at 611 Avenue C, Bayonne, New
Jersey, and Robert Hughes ("Executive").  The term "Bank" refers to Pamrapo
Savings Bank, SLA, the wholly-owned subsidiary of the Holding Company or any
successor thereto.

     WHEREAS, the Holding Company recognizes the substantial contribution
Executive has made to the Holding Company and wishes to protect his position
therewith for the period provided in this Agreement; and

     WHEREAS, Executive has agreed to serve in the employ of the Holding Company
or an affiliate thereof.

     NOW, THEREFORE, in consideration of the contribution and responsibilities
of Executive, and upon the other terms and conditions hereinafter provided, the
parties hereto agree as follows:

1.   TERM OF AGREEMENT.
     ----------------- 

     The term of this Agreement shall be deemed to have commenced as of the date
first above written and shall continue for a period of thirty-six (36) full
calendar months thereafter.  Commencing on January 1, 1997, the term of this
Agreement shall be extended one day each day until such time as the Board of
Directors of the Holding Company (the "Board") or the Executive elects not to
extend the term of this Agreement by giving written notice to the other party in
accordance with Section 4 of this Agreement, in which case the term of this
Agreement shall be fixed and shall end on the third anniversary date of such
written notice.

2.   CHANGE IN CONTROL.
     ----------------- 

     (a) Upon the occurrence of a Change in Control of the Holding Company (as
herein defined) followed at any time during the term of this Agreement by the
termination of Executive's employment, other than for Cause, as defined in
Section 2(c) hereof, the provisions of Section 3 shall apply.  Upon the
occurrence of a Change in Control, Executive shall have the right to elect to
voluntarily terminate his employment at any time during the term of this
Agreement.

     (b) For purposes of this Agreement, a "Change in Control" of the
Association or Holding Company shall mean an event of a nature that: (i) would
be required to be reported in response to Item 1(a) of the Current Report on
Form 8-K, as in effect on the date hereof, pursuant to Section 13 or 15(d) of
the Securities Exchange Act of 1934 (the "Exchange Act"); or (ii) results in a
Change in Control of the Bank or the Holding Company within the meaning of the
Home Owners' Loan Act of 1933 and the Rules and Regulations promulgated
<PAGE>
 
by the Office of Thrift Supervision ("OTS") (or its predecessor agency), as in
effect on the date hereof (provided, that in applying the definition of change
in control as set forth under the rules and regulations of the OTS, the Board
shall substitute its judgment for that of the OTS); or (iii) without limitation
such a Change in Control shall be deemed to have occurred at such time as (A)
any "person" (as the term is used in Sections 13(d) and 14(d) of the Exchange
Act) is or becomes the "beneficial owner" (as defined in Rule 13d-3 under the
Exchange Act), directly or indirectly, of securities of the Bank or the Holding
Company representing 20% or more of the Bank's or the Holding Company's
outstanding securities except for any securities of the Bank purchased by the
Holding Company in connection with the conversion of the Bank to the stock form
and any securities purchased by any employee benefit plan of the Bank, or (B)
individuals who constitute the Board on the date hereof (the "Incumbent Board")
cease for any reason to constitute at least a majority thereof, provided that
any person becoming a director subsequent to the date hereof whose election was
approved by a vote of at least three-quarters of the directors comprising the
Incumbent Board, or whose nomination for election by the Holding Company's
stockholders was approved by the same Nominating Committee serving under an
Incumbent Board, shall be, for purposes of this clause (B), considered as though
he were a member of the Incumbent Board, or (C) a plan of reorganization,
merger, consolidation, sale of all or substantially all the assets of the Bank
or the Holding Company or similar transaction occurs in which the Bank or
Holding Company is not the resulting entity, or (D) a proxy statement is
distributed soliciting proxies from stockholders of the Holding Company, by
someone other than the current management of the Holding Company, seeking
stockholder approval of a plan of reorganization, merger or consolidation of the
Holding Company or Bank with one or more corporations as a result of which the
outstanding shares of the class of securities then subject to such plan or
transaction are exchanged for or converted into cash or property or securities
not issued by the Bank or the Holding Company shall be distributed, or (E) a
tender offer is made for 20% or more of the voting securities of the Bank or
Holding Company then outstanding.

     (c) Executive shall not have the right to receive termination benefits
pursuant to Section 3 hereof upon Termination for Cause.  The term "Termination
for Cause" shall mean termination because of Executive's personal dishonesty,
incompetence, willful misconduct, any breach of fiduciary duty involving
personal profit, intentional failure to perform stated duties, willful violation
of any law, rule, regulation (other than traffic violations or similar offenses)
or final cease and desist order, or any material breach of this Agreement.  In
determining incompetence, the acts or omissions shall be measured against
generally prevailing standards of professional competence for officers having
comparable positions in the savings institutions industry.  Notwithstanding the
foregoing, Executive shall not be deemed to have been Terminated for Cause
unless and until there shall have been delivered to him a copy of a resolution
duly adopted by the affirmative vote of not less than three-fourths of the
members of the Board at a meeting of the Board called and held for that purpose
(after reasonable notice to Executive and an opportunity for him, together with
counsel, to be heard before the Board), finding that in the good faith opinion
of the Board, Executive was guilty of conduct justifying Termination for Cause
and specifying the particulars thereof in detail.  Executive shall not have the
right to receive compensation or other benefits for any period after Termination
for Cause.  Any stock options and related limited rights granted to

                                       2
<PAGE>
 
Executive under any stock option plan, or any unvested awards granted to
Executive under any restricted stock benefit plan of the Holding Company or its
subsidiaries, shall become null and void effective upon Executive's receipt of
Notice of Termination For Cause pursuant to Section 8 hereof, and shall not be
exercisable by or delivered to Executive at any time subsequent to such
Termination For Cause.

3.   TERMINATION BENEFITS.
     -------------------- 

     (a) Upon the occurrence of a Change in Control, followed at any time during
the term of this Agreement by the voluntary or involuntary termination of
Executive's employment, other than for Termination for Cause, the Holding
Company shall be obligated to pay Executive, or in the event of his subsequent
death, his beneficiary or beneficiaries, or his estate, as the case may be, a
sum equal to three (3) times Executive's average annual compensation for the
three (3) preceding taxable years, such annual compensation shall include any
bonuses and any other compensation paid or to be paid to Executive in any such
year, the amount of benefits paid or accrued to Executive pursuant to any
employee benefit plan maintained by the Bank or Holding Company in any such year
and the amount of any contributions made or to be made on behalf of Executive
pursuant to any employee benefit plan maintained by the Bank or the Holding
Company in any such year.  At the election of Executive which election is to be
made prior to a Change in Control, such payment shall be made in a lump sum.  In
the event that no election is made, payment to Executive will be made on a
monthly basis in approximately equal installments during the remaining term of
this Agreement.

     (b) Upon the occurrence of a Change in Control of the Bank or the Holding
Company followed at any time during the term of this Agreement by Executive's
termination of employment, other than for Termination for Cause, the Holding
Company shall cause to be continued life, medical and disability coverage
substantially identical to the coverage maintained by the Bank for Executive
prior to his severance, except to the extent such coverage may be changed in its
application to all Bank employees.  Such coverage and payments shall cease upon
expiration of thirty-six (36) full calendar months following the Date of
Termination.

     (c) Notwithstanding the preceding paragraphs of this Section 3, in the
event that:

               (i) the aggregate payments or benefits to be made or afforded to
          Executive under this Section 3, when taken together with any payments
          or benefits under any other provisions of:  this Agreement; or of any
          other agreement, contract, or understanding heretofore or hereafter
          entered into between the Executive and Pamrapo Savings Bank, SLA or
          Pamrapo Bancorp, Inc. ("Other Agreement(s)"), except an agreement,
          contract, or understanding hereafter entered into that expressly
          modifies or excludes application of this Section 3; or of any formal
          or informal plan or other arrangement heretofore or hereafter adopted
          by Pamrapo Savings Bank, SLA or Pamrapo Bancorp, Inc. for the direct
          or indirect provision of compensation to employees of Pamrapo

                                       3
<PAGE>
 
          Savings Bank, SLA or Pamrapo Bancorp, Inc. (including groups or
          classes of participants or beneficiaries of which the Executive is a
          member), whether or not such compensation is deferred, is in cash, or
          is in the form of a benefit to or for the Executive ("Benefit
          Plan(s)"), (herein referred to collectively as the "Termination
          Benefits") would be deemed to include an "excess parachute payment"
          within the meaning of Section 280G(b) of the Internal Revenue Code of
          1986, as amended, (the"Code") (an "Excess Parachute Payment"), and

               (ii) if such Termination Benefits were reduced to an amount (the
          "Non-Triggering Amount"), the value of which is one dollar ($1.00)
          less than an amount equal to three (3) times Executive's "base
          amount," as determined in accordance with said Section 280G, and the
          Non-Triggering Amount would be greater than the aggregate value of the
          Termination Benefits (without such reduction) minus the amount of tax
          required to be paid by Executive thereon by Section 4999 (or any
          successor provision) of the Code, then the Termination Benefits shall
          be reduced to the Non-Triggering Amount.  The allocation of the
          reduction required by this paragraph of Section 3 shall be determined
          by the Executive.    The Executive shall have the right, in the
          Executive's sole discretion, to designate those payments or benefits
          under this Agreement, any Other Agreements, and/or any Benefit Plans,
          which should be reduced or eliminated so as to avoid having the
          payment to the Executive under this Agreement to be deemed an Excess
          Parachute Payment.
 
4.   NOTICE OF TERMINATION.
     --------------------- 

     (a) Any purported termination by the Holding Company, or by Executive shall
be communicated by Notice of Termination to the other party hereto.  For
purposes of this Agreement, a "Notice of Termination" shall mean a written
notice which shall indicate the specific termination provision in this Agreement
relied upon and shall set forth in detail the facts and circumstances claimed to
provide a basis for termination of Executive's employment under the provision so
indicated.

     (b) "Date of Termination" shall mean the date specified in the Notice of
Termination (which, in the case of Termination for Cause, shall not be less than
thirty (30) days from the date such Notice of Termination is given).

     (c) If, within thirty (30) days after any Notice of Termination is given,
the party receiving such Notice of Termination notifies the other party that a
dispute exists concerning the termination, the Date of Termination shall be the
date on which the dispute is finally determined, either by mutual written
agreement of the parties, by a binding arbitration award, or by a final
judgment, order or decree of a court of competent jurisdiction (the time for
appeal therefrom having expired and no appeal having been perfected) and
provided further that the Date of Termination shall be extended by a notice of
dispute only if such notice is given in good faith and the party giving such
notice pursues the resolution of such dispute with reasonable diligence.
Notwithstanding the pendency of any such dispute, the Holding

                                       4
<PAGE>
 
Company will continue to pay Executive his full compensation in effect when the
notice giving rise to the dispute was given (including, but not limited to his
current annual salary) and continue him as a participant in all compensation,
benefit and insurance plans in which he was participating when the notice of
dispute was given, until the dispute is finally resolved in accordance with this
Agreement.  Amounts paid under this Section 4(c) are in addition to all other
amounts due under this Agreement and shall not be offset against or reduce any
other amounts due under this Agreement.

5.   SOURCE OF PAYMENTS.
     ------------------ 

     It is intended by the parties hereto that all payments provided in this
Agreement shall be paid in cash or check from the general funds of the Holding
Company.  Further, the Holding Company guarantees such payment and provision of
all amounts and benefits due hereunder to Executive and, if any amount and
benefits due from the Bank are not timely paid or provided by the Bank, such
amounts and benefits shall be paid and provided by the Holding Company.

6.   EFFECT ON PRIOR AGREEMENTS AND EXISTING BENEFIT PLANS.
     ----------------------------------------------------- 

     This Agreement contains the entire understanding between the parties hereto
and supersedes any prior agreement between the Holding Company and Executive,
except that this Agreement shall not affect or operate to reduce any benefit or
compensation inuring to Executive of a kind elsewhere provided.  No provision of
this Agreement shall be interpreted to mean that Executive is subject to
receiving fewer benefits than those available to him without reference to this
Agreement.

     Nothing in this Agreement shall confer upon Executive the right to continue
in the employ of the Holding Company or shall impose on the Holding Company any
obligation to employ or retain Executive in its employ for any period.

7.   NO ATTACHMENT.
     ------------- 

     (a) Except as required by law, no right to receive payments under this
Agreement shall be subject to anticipation, commutation, alienation, sale,
assignment, encumbrance, charge, pledge, or hypothecation, or to execution,
attachment, levy, or similar process or assignment by operation of law, and any
attempt, voluntary or involuntary, to affect any such action shall be null,
void, and of no effect.

     (b) This Agreement shall be binding upon, and inure to the benefit of,
Executive, the Holding Company and their respective successors and assigns.

                                       5
<PAGE>
 
8.   MODIFICATION AND WAIVER.
     ----------------------- 

     (a) This Agreement may not be modified or amended except by an instrument
in writing signed by the parties hereto.

     (b) No term or condition of this Agreement shall be deemed to have been
waived, nor shall there be any estoppel against the enforcement of any provision
of this Agreement, except by written instrument of the party charged with such
waiver or estoppel.  No such written waiver shall be deemed a continuing waiver
unless specifically stated therein, and each such waiver shall operate only as
to the specific term or condition waived and shall not constitute a waiver of
such term or condition for the future or as to any act other than that
specifically waived.

9.   SEVERABILITY.
     -------------

     If, for any reason, any provision of this Agreement, or any part of any
provision, is held invalid, such invalidity shall not affect any other provision
of this Agreement or any part of such provision not held so invalid, and each
such other provision and part thereof shall to the full extent consistent with
law continue in full force and effect.

10.  HEADINGS FOR REFERENCE ONLY.
     --------------------------- 

     The headings of sections and paragraphs herein are included solely for
convenience of reference and shall not control the meaning or interpretation of
any of the provisions of this Agreement.

11.  GOVERNING LAW.
     ------------- 

     The validity, interpretation, performance, and enforcement of this
Agreement shall be governed by the laws of the State of Delaware.

12.  ARBITRATION.
     ----------- 

     Any dispute or controversy arising under or in connection with this
Agreement shall be settled exclusively by arbitration, conducted before a panel
of three arbitrators sitting in a location selected by Executive within fifty
(50) miles from the location of the Holding Company, in accordance with the
rules of the American Arbitration Association then in effect.  Judgment may be
entered on the arbitrator's award in any court having jurisdiction; provided,
however, that Executive shall be entitled to seek specific performance of his
right to be paid until the Date of Termination during the pendency of any
dispute or controversy arising under or in connection with this Agreement.

                                       6
<PAGE>
 
13.  PAYMENT OF LEGAL FEES.
     --------------------- 

     All reasonable legal fees paid or incurred by Executive pursuant to any
dispute or question of interpretation relating to this Agreement shall be paid
or reimbursed by the Holding Company if Executive is successful pursuant to a
legal judgment, arbitration or settlement.

14.  INDEMNIFICATION.
     --------------- 

     The Holding Company shall provide Executive (including his heirs, executors
and administrators) with coverage under a standard directors' and officers'
liability insurance policy at its expense, or in lieu thereof, shall indemnify
Executive (and his heirs, executors and administrators) to the fullest extent
permitted under Delaware law and as provided in the Holding Company's
certificate of incorporation against all expenses and liabilities reasonably
incurred by him in connection with or arising out of any action, suit or
proceeding in which he may be involved by reason of his having been a director
or officer of the Holding Company (whether or not he continues to be a director
or officer at the time of incurring such expenses or liabilities), such expenses
and liabilities to include, but not be limited to, judgments, court costs and
attorneys' fees and the cost of reasonable settlements.

15.  SUCCESSOR TO THE HOLDING COMPANY.
     -------------------------------- 

     The Holding Company shall require any successor or assignee, whether direct
or indirect, by purchase, merger, consolidation or otherwise, to all or
substantially all the business or assets of the Bank  or the Holding Company,
expressly and unconditionally to assume and agree to perform the Holding
Company's obligations under this Agreement, in the same manner and to the same
extent that the Holding Company would be required to perform if no such
succession or assignment had taken place.

                                   SIGNATURES

     IN WITNESS WHEREOF, Pamrapo Bancorp, Inc. has caused this Agreement to be
executed by its duly authorized officer, and Executive has signed this
Agreement, on the 25th day of February, 1997.
                 

ATTEST:                               Pamrapo Bancorp, Inc.


 /s/ Margaret Russo                   By: /s/ William J. Campbell 
----------------------------------       ---------------------------------
Secretary                                William J. Campbell
                                         President and Chief Executive   
                                         Officer                          
                                         
WITNESS:                                 

 /s/ Judith M. McAuliffe                  /s/ Robert A. Hughes
----------------------------------       ---------------------------------
                                         Robert A. Hughes
                                         Executive

Seal

                                       7