<PAGE>   1
                                                                  EXHIBIT 10.3.1

                             STOCK PLEDGE AGREEMENT


         AGREEMENT dated this 31st day of March, 1999, by and between BRETT N.
SILVERS, of West Hartford, Connecticut ("PLEDGOR"), and FIRST INTERNATIONAL
BANCORP, INC., a Delaware corporation ("PLEDGEE").

                               W I T N E S E T H:

         WHEREAS, Pledgor is indebted to Pledgee pursuant to a Promissory Note
dated March 31, 1999, in the principal amount of $1,980,000 (the "NOTE"); and

         WHEREAS, such indebtedness is being incurred by Pledgor in connection
with Pledgor's purchase from Pledgee of 200,000 shares of Common Stock of
Pledgee (the "STOCK"); and

         WHEREAS, Pledgor has agreed to pledge the Stock to Pledgee as
collateral for the Note, on the terms set forth herein.

         NOW, THEREFORE, it is hereby agreed as follows:

         1. Pledge. As collateral for the Note and any substitutions or
replacements for the Note (collectively, the "OBLIGATIONS"), Pledgor hereby
pledges, assigns and delivers to the Pledgee and grants to Pledgee a first lien
security interest in the Stock. The Pledgor will deliver or cause to be
delivered to Pledgee, as soon as it is received by Pledgor or becomes available
to Pledgor from the issuer or any transfer agent for the issuer, a certificate
evidencing the ownership of the Stock. Contemporaneously herewith, Pledgor is
delivering to Pledgee a stock power executed in blank with respect to the Stock.
The Pledgee shall hold the Stock as security for the payment of and performance
of the Obligations and Pledgor shall not encumber, assign or dispose of the
Stock except in accordance with the provisions of this Agreement and except that
Pledgor shall have the right to transfer the Stock, subject to the first lien
security interest granted in this Agreement, to one or more family members of
Pledgor and/or trusts as to which the sole beneficiaries are Pledgor and/or
family members of Pledgor. However, Pledgor shall have the right at any time or
times to sell all or any of the Stock, free and clear of the lien of Pledgee, so
long as the proceeds of such sale are applied first to reduce the balance, if
any, of the Obligations.

         2. Dividends And Other Rights. If the Pledgor becomes entitled to
receive or receives cash dividends or any other distribution with respect to the
Stock, Pledgor shall be entitled to retain all of such dividend or distribution.
<PAGE>   2
         3. Representation. The Pledgor warrants and represents that (i) the
Stock is duly and validly pledged to the Pledgee; and (ii) Pledgor has good
title to all the Stock, free and clear of all pledges and other encumbrances.

         4. Stock Adjustments or Additions. In the event that during the term of
this pledge any stock dividend is declared or made, or if any reclassification,
readjustment or other change is made in the capital structure of the Company
(collectively, "STOCK DIVIDEND OR CHANGE"), 100% of all new, substituted and
additional shares or securities of the Company issued to or acquired by Pledgor
by reason of such Stock Dividend or Change shall be forthwith delivered to the
Pledgee to be held by it under this Agreement, and the term "STOCK" shall be
deemed to include such shares or securities.

         5. Term. The pledge shall terminate upon payment of and performance of
all Obligations. Upon termination of this pledge, any Stock still held hereunder
shall be delivered forthwith to the Pledgor.

         6. Default. The Pledgee shall have all of the rights and remedies with
respect to the Stock subject to the Uniform Commercial Code in force in
Connecticut on the date hereof (the "CODE"). Upon the occurrence of an event of
default under the Note, and during the continuation thereof, the Pledgee may (i)
cause all or any part of the Stock to be transferred into its name or into the
name of its nominee or nominees, and (ii) apply the Stock against the payment or
performance of the Obligations. Pledgor agrees that because of the Securities
Act of 1933, as amended, or any other laws or regulations, and for other
reasons, there may be legal and/or practical restrictions or limitations
affecting Pledges in any attempts to dispose of certain portions of the Stock
and for enforcement of its rights. For these reasons, Pledgee is hereby
authorized by Pledgor, but not obligated, in the event of the occurrence of an
event of default under the Documents, to sell all or any part of the Stock at
private sale, subject to investment letter or in any other manner which will not
require the Stock, or any part thereof, to be registered in accordance with the
Securities Act of 1933, as amended, or the rules and regulations promulgated
thereunder, or any other law or regulation, at a commercially reasonable price
obtainable by Pledgee at any such private sale or other disposition in the
manner mentioned above. Pledgee is also hereby authorized by Pledgor, but not
obligated, to take such actions, give such notices, obtain such consents, and do
such other things as Pledgee may deem to be required or appropriate in the event
of sale or disposition of any of the Stock. Pledgee may in its discretion
approach a restricted number of potential purchasers and Pledgor acknowledges
that a sale under such circumstances may yield a lower price for the Stock, or
any part or parts thereof, than would otherwise be obtainable if same were
either offered to a large number of potential purchasers, or registered and sold
in the open market. Pledgor agrees (a) that in the event Pledgee shall upon the
occurrence of an event of default under the Documents (an "EVENT OF DEFAULT"),
sell the Stock, or any portion thereof, at such private sale or sales, Pledgee
shall have the right to rely upon the advice and opinion of any member firm of a
national securities exchange as to the commercially
<PAGE>   3
reasonably price obtainable upon such a private sale thereof, and (b) that such
reliance shall be conclusive evidence that Pledgee handled such matter in a
commercially reasonable manner under the Code.

         7. Voting. Until the occurrence of an event of default under the Note,
Pledgor shall have the exclusive right to vote the Stock, at any and all
meetings of the shareholders of the Company. Following the occurrence of an
event of default under the Note, and during the continuance thereof, Pledgee
shall have the right, after either giving written notice thereof to Pledgor that
it intends thereafter to do so or after transferring the shares into the name of
Pledgee or its nominee, at its sole discretion and without liability therefor,
to cause such shares not to be voted.

         8. Duty of Care. Beyond the exercise of reasonable care to assure the
safe custody of the certificates evidencing Stock while held hereunder, the
Pledgee shall have no duty or liability to collect any sums due in respect
thereof or to protect or preserve rights pertaining thereto, and shall be
relieved of all responsibility for the Stock upon surrendering the same to the
Pledgor.

         9. Remedies Not Exclusive. The rights and remedies herein provided are
cumulative and are in addition to, and not exclusive of, any rights or remedies
provided by law, including without limitation, the rights and remedies of a
secured party under the Uniform Commercial Code in force in Connecticut on the
date hereof and as may be amended from time to time.

         10. Effect. This Agreement shall be binding upon and inure to the
benefit of the parties hereto and their respective successors and assigns.

         11. Applicable Law. This Agreement shall be governed by and construed
according to the laws of the State of Connecticut and may not be amended except
in writing.

         IN WITNESS WHEREOF, the parties hereto have caused this Agreement to be
executed and delivered on the date first above written.


                                   FIRST INTERNATIONAL BANCORP, INC.


                                   By:/s/ Leslie A, Galbraith
                                      ------------------------------
                                       Leslie A. Galbraith
                                       Its Executive Vice President


                                   /s/ Brett N. Silvers
                                   ---------------------------------
                                   Brett N. Silvers