<PAGE>
                                                                 EXHIBIT 10.61.3

                           SECOND AMENDED AND RESTATED

                              STANDSTILL AGREEMENT


     This Second Amended and Restated Standstill Agreement (this "AGREEMENT") is
made as of June 10, 2004 by and between Catalyst Semiconductor, Inc., a
corporation organized and existing under the laws of the State of Delaware
("CATALYST"), Roland Duchatelet and Elex N.V., a corporation organized and
existing under the laws of the Country of Belgium (individually, the "PURCHASER"
and collectively with Mr. Duchaletet, the "Purchasers"). This Agreement amends
and restates that certain Amended and Restated Standstill Agreement dated as of
September 14, 1998 (the "PRIOR AGREEMENT") between Catalyst and Purchaser and
Amendment No. 1 dated April 22, 2004 to the Prior Agreement between the Company
and the Purchaser ("AMENDMENT NO. 1").

                                 R E C I T A L S

     In connection with negotiations relating to the proposed sale by Purchasers
of shares of Catalyst's Common Stock in a firm commitment underwritten public
offering, Purchaser and Catalyst wish to amend and restate the Prior Agreement
and Amendment No. 1 such that: (i) Purchasers makes certain covenants to
Catalyst so as to provide limits on Purchasers' ownership of and voting of
Catalyst's capital stock, (ii) Catalyst permits Purchasers to participate in the
firm commitment underwritten public offering and (iii) Catalyst and Purchasers
agree to a methodology for the orderly disposition of Purchasers' remaining
shares of Catalyst's capital stock held by Purchasers.

     NOW, THEREFORE, in consideration of the foregoing and the mutual promises
herein contained, the parties agree as follows:

                                A G R E E M E N T

     1. Certain Definitions. Unless the context otherwise requires, the
following terms, for all purposes of this Agreement, shall have the meanings
specified in this Section 1:

     "AFFILIATE" of any Person shall mean: (i) any entity which owns fifty
percent (50%) or more of such Person's Outstanding Voting Stock or which has the
power to appoint or elect a majority of the Board of Directors of such Person;
(ii) any entity in which such Person owns fifty percent (50%) or more of such
entity's Outstanding Voting Stock or for which such Person has the power to
appoint or elect a majority of the Board of Directors of such entity; (iii) any
entity having a common stockholder with such Person wherein such stockholder
owns fifty percent (50%) or more of such entity's Outstanding Voting Stock and
has the power to appoint a majority of the Board of Directors of such Person;
(iv) any entity having a common stockholder with such Person wherein such
stockholder owns fifty percent (50%) or more of such Person's Outstanding Voting
Stock and has the power to appoint a majority of the Board of Directors of such
entity; or (v) any entity having a common stockholder with such Person wherein
such stockholder has the power to appoint a majority of the Board of Directors
of such Person and such entity.

     "COMMON STOCK" shall mean the common stock, par value $0.001 per share, of
Catalyst.
<PAGE>

     "OUTSTANDING VOTING STOCK" of Catalyst, or another entity as the context
requires, shall mean the Common Stock of Catalyst (or such other entity) and any
other securities convertible into Common Stock of Catalyst (or such other
entity) having the power to vote in the election of directors of Catalyst (or
such other entity) then outstanding, other than securities having such power
only upon the happening of a contingency which has not yet occurred.

     "PERCENTAGE INTEREST" shall mean the fraction equal to (i) the aggregate
number of shares of Voting Stock held by the Purchasers on such date and any
options or other securities that may be exercised or converted into shares of
Common Stock by the Purchasers within 60 days from the date of such calculation
divided by (ii) the Outstanding Voting Stock of Catalyst on such date plus any
options or other securities that may be exercised or converted into shares of
Common Stock by the Purchasers within 60 days from the date of such calculation.

     "PERSON" shall mean any individual, partnership, joint venture,
corporation, trust or unincorporated organization, or any business entity or
governmental authority, in any case whether acting in an individual, fiduciary
or other capacity.

     "PUBLIC OFFERING" shall mean a firm commitment underwritten public offering
of the Common Stock pursuant to a registration statement of Catalyst declared
effective by the United States Securities and Exchange Commission in which the
Purchasers sell shares of the Common Stock.

     "REGISTRATION STATEMENT" shall mean the registration statement on Form S-3
filed by Catalyst with the United States Securities and Exchange Commission on
June [14], 2004 which includes shares of the Common Stock held by the Purchasers
as a portion of the shares to be sold pursuant to such registration statement.

     "VOTING POWER" shall mean the number of votes entitled to then be cast by
the Outstanding Voting Stock of Catalyst at any election of directors of
Catalyst, PROVIDED that, for the purpose of determining Voting Power, each share
of Preferred Stock of Catalyst, if any (the "PREFERRED STOCK"), shall be deemed
to be entitled to the number of votes equal to the number of shares of Common
Stock into which such share of Preferred Stock could then be converted.

     "VOTING STOCK" shall mean the Common Stock outstanding, assuming the
exercise or conversion of all outstanding securities convertible into or
exercisable or exchangeable for Common Stock and further assuming the issuance
of all shares of Common Stock reserved for issuance pursuant to stock options
whether or not yet granted, and any other securities issued by Catalyst having
the power to vote in the election of directors of Catalyst, other than
securities having such power only upon the happening of a contingency which has
not yet occurred.

     2.   Covenants of Purchaser.

          2.1  Standstill Provisions. Until the later of (i) one year from the
date that the United States Securities and Exchange Commission declares the
Registration Statement effective or (ii) Mr. Duchatelet ceases to be a member of
the Board of Directors of Catalyst, Purchasers shall not (and shall not permit
any Affiliate to), directly or indirectly,
<PAGE>

               (a)  acquire from other holders of Voting Stock or other
securities convertible into or exchangeable or exercisable for Voting Stock,
beneficial ownership of any Voting Stock, any securities convertible into or
exchangeable for Voting Stock, or any other right to acquire Voting Stock
(except, in any case, by way of stock dividends, stock splits or other
distributions made to holders of any Voting Stock generally), or authorize or
make a tender, exchange or other offer which would result in such an
acquisition, without the prior written consent of Catalyst approved by the Board
of Directors of Catalyst, if the effect of such acquisition would be to increase
the Voting Power of all Voting Stock then beneficially owned by Purchaser (and
its Affiliates), or which Purchaser (and its Affiliates) collectively have a
right to acquire, to an amount greater than a 4.99% Percentage Interest in the
Voting Stock inclusive of any exercisable options to acquire Voting Stock of
Catalyst granted to Roland Duchatelet in his capacity as a director of Catalyst;

               (b)  deposit any shares of Voting Stock in a voting trust or
subject any Voting Stock to any arrangement or agreement with respect to the
voting of such Voting Stock;

               (c)  otherwise act to seek to control the management, Board of
Directors, or policies of Catalyst (or any of its Affiliates), or, except as
required by law, make public statements with respect thereto;

               (d)  solicit, or induce others to solicit, "proxies" to vote with
respect to any securities issued by Catalyst or any of its Affiliates, or become
a "participant" or induce others to become a "participant" in any solicitation
of "proxies" to vote (as such terms are used in Regulation 14A under the
Securities Exchange Act of 1934) securities issued by Catalyst (or any of its
Affiliates) or seek to advise or influence any Person with respect to the voting
of securities issued by Catalyst (or its Affiliates); or

               (e)  otherwise act in any manner, whether by forming, joining or
participating with any Person or group, or aiding, advising, encouraging or
assisting any other Person or group, whether for the purpose of acquiring,
holding, voting or disposing of Voting Stock or accomplishing any of the
foregoing Section (a) through (d) above, including inviting, encouraging,
soliciting or providing information to assist the submission of any proposal for
the acquisition, by purchase or otherwise, from Catalyst (or any of its
Affiliates) or any other Person of securities issued by Catalyst (or any of its
Affiliates) or assets of Catalyst (or any of its Affiliates) or any proposal
with respect to any form of restructuring, recapitalization or similar
transaction of Catalyst (or any of its Affiliates).

          2.2  Notice of Purchases. Purchasers shall notify Catalyst in writing
as to Purchasers' (or any Affiliate's) acquisition of additional shares of
Voting Stock, or rights thereto within a reasonable time period not to exceed
ten (10) days after such acquisition. Purchasers shall submit any Voting Stock
purchased after the date of this Agreement to Catalyst's transfer agent for
application of the legend as required by Section 3(b) hereof within 10 days of
the acquisition of the Voting Stock.

          2.3  Transferees. Purchasers shall not sell, assign or transfer any
Common Stock to any other party (or any Affiliate of such party) in a
transaction or a series of related transactions

<PAGE>

without such other party which purchases such shares having agreed to and having
become a party to this Agreement with the Company except as follows:

               (a)  the sale of Common Stock by the Purchasers in the Public
Offering; and

               (b)  after the date that is one year from the date the
Registration Statement is declared effective by the Securities and Exchange
Commission, subject to a sale in compliance with Rule 144 promulgated under the
Securities Act of 1933, as amended.

     3.   Prohibited Transfer.

          3.1  Any purchase which causes Purchasers to be in violation of the
terms of Section 2 above ("PROHIBITED TRANSFER") shall not be effected by
Catalyst and shall be voidable at the option of Catalyst by their giving written
notice to the transferor, his transferee and Purchasers.

          3.2  During the time that Section 2 above is effective, each
certificate representing Common Stock held by Purchasers shall be endorsed by
Catalyst with a legend reading as follows:

     "THE SHARES EVIDENCED HEREBY ARE SUBJECT TO A STANDSTILL AGREEMENT BY AND
     BETWEEN THE COMPANY AND THE HOLDER HEREOF (A COPY OF WHICH MAY BE OBTAINED
     FROM THE COMPANY), AND NO TRANSFER OF THE SHARES EVIDENCED HEREBY SHALL BE
     EFFECTIVE EXCEPT IN COMPLIANCE WITH THE TERMS THEREOF."

     Purchasers shall submit within 30 days of the date of this Agreement, all
certificates representing Common Stock held by them for application of the above
legend. Purchasers also agree that any Common Stock acquired by the exercise of
options held by Mr. Duchatelet shall be in certificate form and shall bear the
legend set forth above.

     4.   Miscellaneous.

          4.1  Governing Law; Jurisdiction. This Agreement shall be governed by
and construed and enforced in accordance with the internal laws of the State of
Delaware, and shall be binding upon the parties hereto in the United States and
worldwide. The federal and state courts within County of Santa Clara in the
State of California shall have exclusive jurisdiction to adjudicate any dispute
arising out of this Agreement.

          4.2  Successors and Assigns. Except as otherwise expressly provided
herein, the provisions hereof shall inure to the benefit of, and be binding
upon, the successor and assigns of the parties hereto.

          4.3  Amendment and Restatement of Former Agreement. If this Agreement
becomes effective, the parties intend that this Agreement amend and restate the
Prior Agreement and Amendment No. 1. If this Agreement becomes effective, the
Prior Agreement and Amendment No. 1 shall be of no further force or effect.
<PAGE>

          4.4  Effect of this Agreement. This Agreement shall be effective
immediately prior to the time the Registration Statement is declared effective
by the Untied States Securities and Exchange Commission. This Agreement shall be
of no force or effect unless the Public Offering occurs.

          4.5  Entire Agreement; Amendment. This Agreement constitutes the full
and entire understanding and agreement between the parties with regard to the
subjects hereof and thereof. Neither this Agreement nor any provision hereof may
be amended, changed, waived, discharged or terminated other than by a written
instrument signed by the party against who enforcement of any such amendment,
change, waiver, discharge or termination is sought. Any amendment or waiver
effected in accordance with this section shall be binding upon each holder of
any securities purchased under this Agreement at the time outstanding (including
securities into which such securities have been converted), each future holder
of all such securities and Catalyst.

          4.6  Notices, etc. All notices and other communications required or
permitted hereunder shall be effective upon receipt and shall be in writing and
may be delivered in person, by telecopy, electronic mail, express delivery
service or U.S. mail, in which event it may be mailed by first-class, certified
or registered, postage prepaid, addressed, to the party to be notified, at the
respective addresses set forth below, or at such other address which may
hereinafter be designated in writing:

                (a) If to Elex N.V., to:

                    Elex N.V.
                    Transportstraat 1
                    B 3980
                    Tessenderlo, Belgium
                    Attention: Chairman of the Board
                    Phone: 011+32 57 22 61 31
                    Fax:   011+32 13 67 21 34

                (b) If to Roland Duchatelet, to:

                    c/o Elex N.V.
                    Transportstraat 1
                    B 3980
                    Tessenderlo, Belgium
                    Phone: 011+32 57 22 61 31
                    Fax:   011+32 13 67 21 34

                (c) If to Catalyst, to:

                    Catalyst Semiconductor, Inc.
                    1250 Borregas Avenue
                    Sunnyvale, California  94089
                    Attention: President and Chief Executive Officer
                    Phone: (408) 542-1090
                    Fax:   (408) 542-1406
<PAGE>

          4.7  Severability. If any provision of this Agreement shall be
judicially determined to be invalid, illegal or unenforceable, the validity,
legality and enforceability of the remaining provisions shall not in any way be
affected or impaired thereby.

          4.8  Titles and Subtitles. The titles of the Articles and Sections of
this Agreement are for convenience of reference only and in no way define,
limit, extend, or describe the scope of this Agreement or the intent of any of
its provisions.

          4.9  Counterparts. This Agreement may be executed in any number of
counterparts, each of which shall be an original, but all of which together
shall constitute one instrument.

          4.10 Delays or Omissions. It is agreed that no delay or omission to
exercise any right, power or remedy accruing to any party upon any breach or
default of any other party under this Agreement shall impair any such right,
power or remedy, nor shall it be construed to be a waiver of any such breach or
default, or any acquiescence therein, or of any similar breach or default
thereafter occurring; nor shall any waiver of any single breach or default be
deemed a waiver of any other breach or default theretofore or thereafter
occurring. It is further agreed that any waiver, permit, consent or approval of
any kind or character of any breach or default under this Agreement, or any
waiver of any provisions or conditions of this Agreement must be in writing and
shall be effective only to the extent specifically set forth in writing, and
that all remedies, either under this Agreement, by law or otherwise, shall be
cumulative and not alternative.

          4.11 Consents. Any permission, consent, or approval of any kind or
character under this Agreement shall be in writing or detrimentally relied upon
and proved by clear and convincing evidence (other than alleged reliance) and
shall be effective only to the extent specifically set forth in such writing or
proved.

          4.12 Payment of Fees and Expenses. Each party shall be responsible for
paying its own fees, costs and expenses in connection with this Agreement and
the transactions herein contemplated.

          4.13 Construction of Agreement. No provision of this Agreement shall
be construed against either party as the drafter thereof.

          4.14 Section References. Unless otherwise stated, any reference
contained herein to a Section or subsection refers to the provisions of this
Agreement.

          4.15 Variations of Pronouns. All pronouns and all variations thereof
shall be deemed to refer to the masculine, feminine, or neuter, singular or
plural, as the context in which they are used may require.

                  [REMAINDER OF PAGE LEFT BLANK INTENTIONALLY]
<PAGE>
     IN WITNESS WHEREOF, the parties have caused this Second Amended and
Restated Standstill Agreement to be duly executed and delivered by their proper
and duly authorized officers as of the day and year first written above.

                               CATALYST SEMICONDUCTOR, INC.



                               By: /s/ Gelu Voicu
                                   -------------------------------------
                                   Gelu Voicu
                                   President and Chief Executive Officer


                               ELEX N.V.



                               By: /s/ Roland Duchatelet
                                   -------------------------------------
                                   Roland Duchatelet
                                   Chairman of the Board


                               ROLAND DUCHATELET


                               /s/ Roland Duchatelet
                               -------------------------------------