<PAGE>
 
                                                                    EXHIBIT 10.3


                                PLEDGE AGREEMENT

     This Pledge Agreement is made, as of the 1st day of October, 1997, by and
among Stephen R. Quehl, an individual residing at 3 Wingate Lane, Acton, MA
01720 (the "Pledgor"), and Gensym Corporation, a Delaware corporation having a
place of business at 125 CambridgePark Drive, Cambridge, MA 02140 (the
"Pledgee").

                                  WITNESSETH:

     WHEREAS, Pledgor has executed and delivered a secured promissory note (the
"Note") of even date herewith in the original principal amount of $100,000
payable to the Pledgee; and

     WHEREAS, the Pledgor is the owner of the shares of common stock of the
Pledgee, as set forth on Exhibit A; and
                         ---------     

     WHEREAS, as collateral security for the obligations of the Pledgor under
the Note, the Pledgor has agreed to pledge and grant to the Pledgee a first
priority security interest in the shares set forth on Exhibit A (the "Pledged
                                                      ---------              
Shares"), as more fully set forth herein;

     NOW THEREFORE, the parties hereto agree and acknowledge that the foregoing
recitals are true and correct and to the following:

     1.   Pledge of Collateral.  As collateral security for the performance of
          --------------------                                                
the obligations of the Pledgor under the Note (the "Obligations"), the Pledgor
hereby pledges and grants to the Pledgee a security interest in and to all of
the Pledged Shares held by the Pledgor, as identified in Exhibit A annexed
                                                         ---------        
hereto, along with any and all stock rights, powers and other distributions,
dividends or proceeds thereof (the "Shares").  In addition, any stock rights,
dividends, powers or other distributions or proceeds received by the Pledgor
shall be held in trust for and delivered to the Pledgee to be held in accordance
with the terms of this Agreement, and shall be included in the Shares described
above.

     2.   Delivery of the Shares.  The Shares have been delivered to the Pledgee
          ----------------------                                                
on the date hereof, together with undated stock powers executed in blank.  Upon
payment in full of the Note, or such lesser portion of the Note as is owed, the
Pledgee shall return to the Pledgor the Shares, undated stock powers as well as
such other instruments, documents, stock certificates, money and goods as may
come into Pledgee's possession from time to time, whether through delivery by
Pledgor or otherwise.
 
     3.   Pledgee's Rights and Duties with Respect to the Collateral.  Pledgee's
          ----------------------------------------------------------            
only duty with respect to the Shares shall be to exercise reasonable care to
secure the safe 

<PAGE>
 
custody thereof all other duties being hereby expressly disclaimed. Pledgee
shall have the right, but not the obligation, to (a) demand, sue for, receive
and collect all money or money damages payable on account of any Shares, (b)
protect, preserve or assert any other rights of Pledgor or take any other action
with respect to the Shares, and (c) pay any taxes, liens, assessments, insurance
premiums or other charges pertaining to Shares. Any expenses incurred by Pledgee
under the preceding sentence shall be paid by Pledgor upon demand, become part
of the Obligations secured by the Shares and bear interest at the per annum rate
equal to the Default Rate set forth in the Note until paid. Pledgee shall be
relieved of all responsibility for the Shares upon surrendering them to Pledgor.
 
     4.   Pledgor's Warranties and Indemnity.
          ---------------------------------- 

          4.1  Pledgor represents, warrants and covenants (a) that Pledgor is
the lawful owner of the Shares, (b) that the Shares are fully paid and
nonassessable, (c) that as of the date hereof, the Shares are free and clear of
all liens, encumbrances, and security interests, other than the security
interest granted by the Pledgor hereunder, and this pledge constitutes a valid
and perfected security interest in the Shares enforceable against the Pledgor,
(d) that the Shares are not subject to any outstanding rights of redemption or
options to purchase or sell, (e) that the Pledgor has the sole right and lawful
authority to pledge the Shares and otherwise to comply with the provisions
hereof, (f) no litigation is pending or threatened against the Pledgor, which if
adversely determined, would have a material adverse effect against the Pledgor
or the Pledgee's rights in respect of the Shares, (g) that the Pledgor agrees to
defend the Pledgee's title in the Shares and the security interest therein
against any and all claims and demands, and (h) this Pledge Agreement
constitutes the legal, valid and binding obligation of the Pledgor, enforceable
against the Pledgor in accordance with its terms.

          4.2  If any adverse claim is asserted in respect of the Shares or any
portion thereof, except as such may arise from the wanton or reckless acts of
the Pledgee, the Pledgor agrees to indemnify the Pledgee and hold the Pledgee
harmless from and against any reasonable liabilities or damages, and reasonable
attorney's fees incurred by the Pledgee in exercising any right, power or remedy
of the Pledgee hereunder.  Any such loss, liability or expense so incurred shall
be paid by the Pledgor upon demand, become part of the Obligations secured by
the Shares and bear interest at the per annum rate equal to the Default Rate (as
defined in the Note).
 
     5.   Voting of Collateral.  While Pledgor is not in default hereunder,
          --------------------                                             
Pledgor may vote the Shares, provided that said voting shall be in conformity
with the Pledgor's performance under this Agreement and the Note.
 
     6.   Dividends and Other Distributions.  While Pledgor is not in default
          ---------------------------------                                  
hereunder, Pledgor may receive all cash dividends, payments of principal and
interest, and other distributions payable with respect to Shares, provided,
                                                                  -------- 
however, that Pledgor 
-------                                                                     

                                      -2-
<PAGE>
 
shall immediately inform Pledgee of the receipt of any such dividend, payment or
other distribution and shall hold the amount thereof in trust for Pledgee unless
and until Pledgee shall in writing release Pledgor from such trust. Pledgor
shall cause all non-cash dividends and distributions with respect to Shares to
be distributed directly to Pledgee, to be held by Pledgee as additional Shares,
and if any such distribution is made to Pledgor he shall receive such
distribution in trust for Pledgee and shall immediately transfer it to Pledgee.
 
     7.   Pledgor's Default.  Pledgor shall be in default hereunder upon the
          -----------------                                                 
occurrence of any of the following events:
 
          7.1  Demand shall have been made or any event of default shall occur
under the Note;

          7.2  If any lien, encumbrance or adverse claim of any nature
whatsoever is asserted with respect to any Shares;
 
          7.3  If any representation or warranty of Pledgor hereunder is or
shall become false; or
 
          7.4  If Pledgor fails to fulfill any obligation hereunder.

     8.   Pledgee's Rights upon Default.  Upon the occurrence of any default as
          -----------------------------                                        
defined in Section 7 hereof, Pledgee may, if Pledgee so elects in its sole
discretion, take any one or more of the following:
 
          8.1  At any time and from time to time sell, assign and deliver all or
any part of the Shares, or any interest therein, at any public or private sale,
for cash, on credit or for other property, for immediate or future delivery
without any assumption of credit risk, and for such price or prices and on such
terms as Pledgee in its absolute discretion may determine; provided that (i) at
                                                           --------            
least ten (10) days' notice of the time and place of any such sale shall be
given to Pledgor, and (ii) in the case of any private sale, such notice shall
also contain the terms of the proposed sale and Pledgee shall sell the Shares
proposed to be sold to any purchaser procured by Pledgor who is ready, willing
and able to purchase, and who prior to the time of such sale tenders the
purchase price of, such Shares on terms more favorable to Pledgee than the terms
contained in such notice; provided, further, the Pledgor acknowledges that the
                          --------  -------                                   
Pledgee may be unable to effect a public sale of all or part of the Shares by
reason of certain prohibitions contained in the Securities Act of 1933, as
amended, and may be compelled to resort to one or more private sales to a
restricted group of purchasers who will be obligated to agree, among other
things, to acquire such securities for their own account, for investment, and
not with a view to the distribution or resale thereof.  The Pledgor acknowledges
that any such private sale may be at prices and on terms less favorable to the
seller than if sold at public sales and that private sales shall be deemed to be
made in a commercially 

                                      -3-
<PAGE>
 
reasonable manner notwithstanding that such a private sale may result in a lower
sale price.
 
          8.2  Exercise the right to vote, the right to receive cash dividends
and other distributions, and all other rights with respect to the Shares as
though Pledgee were the absolute owner thereof, whether or not such rights were
retained by Pledgor as against Pledgee before default; and
 
          8.3  Exercise all other rights available to a secured party under the
Uniform Commercial Code and other applicable law.

          8.4  The rights and remedies available pursuant to this Agreement are
cumulative, and not exclusive of any other rights or remedies otherwise
available to the Pledgee.
 
     9.   Application of Sale Proceeds.  In the event of a sale of Shares, the
          ----------------------------                                        
proceeds shall first be applied to the payment of the expenses of the sale,
including brokers' commissions, counsel fees, any taxes or other charges imposed
by law upon the Shares or the transfer thereof and all other charges paid or
incurred by Pledgee pertaining to the sale; and, second, to satisfy outstanding
Obligations, in the order in which Pledgee elects in its sole discretion; and,
third, the surplus (if any) shall be paid to Pledgor.
 
     10.  Notices.  All notices made or required to be made hereunder shall be
          -------                                                             
sent by United States first class or certified or registered mail, with postage
prepaid, or delivered by hand to Pledgee or to Pledgor at the addresses first
above written.  Notice by mail shall be deemed to have been made on the date
when the notice is deposited in the mail.
 
     11.  Heirs, Successors, Etc. This Agreement and all of its terms and
          ----------------------                                         
provisions shall benefit and bind the heirs, successors, assigns, transferees,
executors and administrators of each of the parties hereto.

     12.  Pledgee's Forbearance.  Any forbearance, failure or delay by Pledgee
          ---------------------                                               
in exercising any right, power or remedy hereunder shall not be deemed a waiver
of such right, power or remedy.  Any single or partial exercise of any right,
power or remedy of Pledgee shall continue in full force and effect until such
right, power or remedy is specifically waived in writing by Pledgee.
 
     13.  Further Assurances.  The Pledgor covenants and agrees to execute and
          ------------------                                                  
deliver, or cause to be executed or delivered, all such other stock powers,
proxies, instruments, and documents, and will take such other action or actions
as the Pledgee may reasonably request from time to time in order to carry out
the provisions and purposes hereof.

                                      -4-
<PAGE>
 
     14.  Termination.  This Agreement and the pledge and security interest
          -----------                                                      
represented hereby shall terminate upon the indefeasible payment in full of the
Obligations.

     15.  Miscellaneous.
          ------------- 

          (a) This Agreement or any part thereof cannot be changed, waived, or
amended except by an instrument in writing signed by Pledgee; and waiver on one
occasion shall not operate as a waiver on any other occasion.

          (b) The Uniform Commercial Code and other laws of the Commonwealth of
Massachusetts shall govern the construction and enforcement of this Agreement.

          (c) If any part of this Agreement or any agreement, document, or
instrument executed in connection herewith shall be deemed invalid or
unenforceable by a court of competent jurisdiction, the remaining provisions
shall remain in full force and effect, and shall continue to be binding upon the
parties.

          (d) This Agreement may be executed in one or more counterparts, each
of which shall constitute an original, but all of which, when taken together,
shall constitute one and the same instrument.

     16.  Jurisdiction.  The Pledgor irrevocably submits to the jurisdiction of
          ------------                                                         
the courts of the Commonwealth of Massachusetts and the United States District
Court for the District of Massachusetts for the purpose of any suit, action or
other proceeding brought by the Lender arising out of or relating to this
Agreement, and the Pledgor waives and agrees not to assert by way of motion, as
a defense or otherwise in any such suit, action or proceeding, any claim that
the Pledgor is not personally subject to the jurisdiction of the courts of the
Commonwealth of Massachusetts or the United States District Court for the
District of Massachusetts or that the Pledgor's property is exempt or immune
from execution or attachment, either prior to judgment or in aid of execution,
that the suit, action or proceeding is brought in an inconvenient forum or that
the venue of the suit, action or proceeding is improper, or that this Pledge
Agreement or the subject matter hereof may not be enforced in or by such court.

                                      -5-
<PAGE>
 
     THE PLEDGOR HEREBY WAIVES ANY AND ALL RIGHTS TO A TRIAL BY JURY IN ANY
ACTION OR PROCEEDING IN CONNECTION WITH ALL MATTERS CONTEMPLATED HEREBY AND
DOCUMENTS EXECUTED IN CONNECTION HEREWITH.

     EXECUTED under seal as of the date first above written.

                              PLEDGOR:

                              /s/ Stephen R. Quehl
                              __________________________________________
                              Stephen R. Quehl


                              PLEDGEE:

                              GENSYM CORPORATION


                              By:  /s/ Lowell B. Hawkinson
                                  ________________________________________
                              Its: Chairman and Chief Executive Officer
                                  ________________________________________

                                      -6-
<PAGE>
 
                                                                    Exhibit 10.3
 
                                   EXHIBIT A
                                       TO
                                PLEDGE AGREEMENT
                             DATED OCTOBER 1, 1997
<TABLE> 
<CAPTION> 

Description                   Certificate No.                No. of Shares
-----------                   ---------------                -------------
<S>                          <C>                            <C> 
Gensym Corporation                                               17,000
Common Stock

</TABLE> 

                                      -7-