<PAGE>
                                                                    Exhibit 4.40



                                               Confidential Treatment Requested:
 The portions of this document marked by "xxxxx" have been omitted pursuant to a
      request for confidential treatment and have been filed separately with the
                                              Securities and Exchange Commission

                         FIRST SUPPLEMENTAL AGREEMENT TO
                     THE REFUNDABLE CROSS DEPOSIT AGREEMENT

This First Supplemental Agreement (the "Supplemental Agreement") is made and
entered into effective as of December 19, 2003 between

CHARTERED SEMICONDUCTOR MANUFACTURING LTD ("Chartered"), with its principal
place of business at 60 Woodlands Industrial Park D, Street 2, Singapore 738406
(hereinafter referred to as "Chartered"),

And

INTERNATIONAL BUSINESS MACHINES CORPORATION ("IBM"), having an office for the
conduct of business at 2070 Route 52, Hopewell Junction, NY 12533, U.S.A.

and is supplemental to the Refundable Cross Deposit Agreement made and entered
into between such parties and effective the 26th November 2002 (the "Refundable
Cross Deposit Agreement").

Chartered and IBM shall hereinafter be jointly referred to as the "Parties" or
individually as a "Party".

NOW THEREFORE, the parties agree as follows:

1.   DEFINITIONS

     Initial capitalized expressions in this Supplemental Agreement shall,
     unless the context otherwise requires, bear the same meaning and
     construction as in the Refundable Cross Deposit Agreement. In addition for
     the purpose of this Supplemental Agreement, the words below in bold letters
     have the following meanings:

     "IBM'S SUPPLEMENTAL PREPAID CAPACITY PERIOD" means the period commencing on
     the first day of the calendar quarter in which IBM first purchases from
     Chartered more than xxxxx wafers under all agreements collectively, and
     ending three (3) years thereafter, or, if Section 4 is applicable, four (4)
     years thereafter.


2.   IBM'S SUPPLEMENTAL PREPAID CAPACITY DEPOSIT

     IBM shall, pursuant to Section 3.5 of the Refundable Cross Deposit
     Agreement, make an IBM's Supplemental Capacity Deposit in the amount of
     xxxxx. The first xxxxx of such IBM's Supplemental Capacity Deposit shall be
     made on or before xxxxx and the balance of such IBM's Supplemental Capacity
     Deposit shall be made on or before xxxxx.


3.   PRICING OF WAFERS

     Notwithstanding Section 2.2 and Exhibit A of the Refundable Cross Deposit
     Agreement:


<PAGE>

(a)  during the IBM's Supplemental Prepaid Capacity Period, and provided that
     IBM's Prepaid Capacity Deposit has not yet been fully refunded pursuant to
     Section 3.4 of the Refundable Cross Deposit Agreement, the price of the
     first xxxxx wafers supplied by Chartered to IBM under all agreements
     collectively in each calendar quarter shall be determined in accordance
     with xxxxx, subject however, in the case of products supplied by Chartered
     to IBM under the IBM Fabrication Subcontract Agreement between the Parties
     and effective as of December 19, 2003, to the provisions on pricing in such
     IBM Fabrication Subcontract Agreement between the Parties and effective as
     of December 19, 2003; for the avoidance of doubt, the refund provisions of
     Section 3.4 of the Refundable Cross Deposit Agreement shall apply to each
     of these first xxxxx wafers;

(b)  during the IBM's Supplemental Prepaid Capacity Period, the price of the
     next xxxxx wafers supplied by Chartered to IBM under all agreements
     collectively in each calendar quarter shall be xxxxx, subject however, in
     the case of products supplied by Chartered to IBM under the IBM Fabrication
     Subcontract Agreement between the Parties and effective as of December 19,
     2003, to the provisions on pricing in such IBM Fabrication Subcontract
     Agreement between the Parties and effective as of December 19, 2003; for
     the avoidance of doubt, the refund provisions of Section 3.4 of the
     Refundable Cross Deposit Agreement shall not apply to each of these next
     xxxxx wafers; for the further avoidance of doubt, if IBM's Prepaid Capacity
     Deposit has been fully refunded pursuant to Section 3.4 of the Refundable
     Cross Deposit Agreement, then this subparagraph (b) shall, instead, apply
     to the first xxxxx wafers supplied by Chartered to IBM under all agreements
     collectively in each calendar quarter falling in IBM's Supplemental Prepaid
     Capacity.

(c)  during IBM's Supplemental Prepaid Capacity Period, the price of all
     additional wafers supplied by Chartered to IBM under all agreements
     collectively in each calendar quarter shall be determined in accordance
     with xxxxx, subject however, in the case of products supplied by Chartered
     to IBM under the IBM Fabrication Subcontract Agreement between the Parties
     and effective as of December 19, 2003, to the provisions on pricing in such
     IBM Fabrication Subcontract Agreement between the Parties and effective as
     of December 19, 2003; for the avoidance of doubt, (i) the refund provisions
     of Section 3.4 of the Refundable Cross Deposit Agreement shall not apply to
     any of such additional wafers, and (ii) IBM shall not be obliged to make
     any IBM's Supplemental Prepaid Capacity Deposit in connection with any of
     such additional wafers.


4.   EXTENSION OF IBM'S SUPPLEMENTAL PREPAID CAPACITY PERIOD

     In the event that, after three (3) years have lapsed from the first day of
     the calendar quarter period in which IBM first purchases from Chartered
     more than xxxxx wafers under all agreements collectively, IBM has purchased
     from Chartered less than xxxxx wafers in that three (3)-year period at the
     price(s) determined in accordance with Section 3(b) of this Supplemental
     Agreement, the IBM's Supplemental Prepaid Capacity Period shall be
     continued for a further one (1) year. During such fourth year of IBM's
     Supplemental Prepaid Capacity Period, if applicable, if IBM desires to
     purchase, at a price of xxxxx, more than xxxxx wafers to complete purchase
     of a cumulative total of xxxxx wafers at such price, then IBM and Chartered
     shall discuss and then agree on a mutually acceptable solution.


5.   REFUNDABILITY OF DEPOSIT

5.1  In the event that either the IBM Fabrication Subcontract Agreement between
     the Parties and effective as of December 19, 2003 or the Manufacturing
     Agreement

<PAGE>

     No.001254 made between the Parties and effective as of January 31, 2003 is
     lawfully terminated by IBM, or in the event the Refundable Cross Deposit
     Agreement is lawfully terminated by IBM under Sections 8.1(b), 8.1(c),
     8.1(f), 8.1(g) or 8.1(i), then the IBM's Supplemental Capacity Deposit made
     pursuant to this Supplemental Agreement shall be refunded by Chartered to
     the following extent:

     R = xxxxx - N  x  xxxxx
         ---------
           xxxxx

     where

     "R" is the dollar value of the portion of IBM's Supplemental Prepaid
     Capacity Deposit to be refunded; and

     "N" is the aggregate number of wafers sold by Chartered to IBM during the
     IBM's Supplemental Prepaid Capacity Period at the price(s) determined in
     accordance with Section 3(b) of this Supplemental Agreement. For purposes
     of this equation, the maximum value for N is xxxxx.


6.   INCORPORATION BY REFERENCE

     Sections 8, 9, 10, 11,12 and 13 of the Refundable Cross Deposit Agreement
     are incorporated by reference into this Supplemental Agreement and shall
     apply to this Supplemental Agreement as if set out in full herein.



IN WITNESS WHEREOF the Parties have hereunto entered into this Supplemental
Agreement as at the date first above written.



/s/ Chia Song Hwee
------------------
Name:  Chia Song Hwee
Title: President and Chief Executive Officer
       for and on behalf of
       CHARTERED SEMICONDUCTOR MANUFACTURING LTD



/s/ Dr. John Kelly, III
-----------------------
Name:  Dr John Kelly, III
Title: Senior Vice President and Group Executive, Technology Group
       for and on behalf of
       INTERNATIONAL BUSINESS MACHINES CORPORATION