<PAGE>
 
                                                                   EXHIBIT 10.21


                                 SEVERANCE AGREEMENT



THIS AGREEMENT (the "Agreement") is made and entered into as of the ninth day of
June, 1998, by and between Certified Grocers of California, Ltd., a California
corporation (the "Company" or "Certified") and Robert M. Ling, Jr. (the
"Executive").

WHEREAS, the Company considers the establishment and maintenance of a sound and
vital management to be essential to protecting and enhancing the best interests
of the Company and its shareholders; and

WHEREAS, the Executive has been determined to be a vital member of the senior
management team of the Company; and

WHEREAS, the Company recognizes that the possibility of termination due to
Change of Control or without cause may arise and the uncertainty it may raise
among management of the Company may result in the departure or distraction of
management personnel, in each case to the detriment of the Company and its
shareholders.

NOW, THEREFORE, in consideration of the promises and of the mutual covenants
herein contained, it is agreed as follows:

This Agreement sets forth the severance benefits which the Company agrees will
be provided to the Executive in the event the Executive's employment with the
Company is terminated without cause under the circumstances described below.


     1.   RIGHT TO TERMINATE.  Certified or the Executive may terminate the
          ------------------                                               
          Executive's employment at any time, subject to Certified's providing
          the benefits hereinafter specified in accordance with the terms
          thereof.


     2.   TERM OF AGREEMENT.  This Agreement shall commence on June 9, 1998; and
          -----------------                                                     
          shall continue in effect until the earlier of June 9, 2001, or
          termination of the Executive's employment; provided, however, that
          commencing on June 9, 2001 the term of this Agreement shall be
          extended for one additional year unless at least 90 days prior to such
          date, Certified or the Executive shall have given written notice per
          Section 9 herein that this Agreement shall not be extended; and
          provided, further, that notwithstanding the delivery of any such
          notice, the term of this Agreement shall automatically extend for an
          additional period of twelve (12) months after a Change of Control of
          the Company.  For the purposes of this Agreement, "Change of Control"
          shall mean (i) The acquisition by any person, entity or group within
          the meaning of Section 13(d) or 14(d) of the Securities and Exchange
          Act of 1934, of beneficial ownership of fifty-one percent (51%) or
          more of the outstanding Class A Shares of Certified Grocers of
          California, Ltd.; or (ii) if the individuals who presently serve on
          the Certified 

<PAGE>
 
          Board of Directors no longer constitute a majority of the members of
          the Certified Board of Directors; provided, however, any person who
          becomes a director subsequent to the commencement date of this
          Employment Agreement, who was elected to fill a vacancy by a majority
          of Certified's members shall be considered as if a member prior to the
          commencement date of this Agreement; or (iii) A liquidation or
          dissolution of Certified or the sale of all or substantially all of
          the assets of Certified.


     3.   TERMINATION.  The Executive shall be entitled to the benefits provided
          -----------                                                           
          in Section 4 hereof upon the termination of the Executive's employment
          with Certified during this Agreement period, unless such termination
          is (i) because of the Executive's death, Disability, or Retirement,
          (ii) by Certified for Cause, or (iii) by the Executive other than for
          Good Reason (as all such capitalized terms are hereinafter defined).


                    (a)  Disability.  Termination by Certified of the 
                         ----------
               Executive's employment based on "Disability" shall mean the
               Executive's incapacity due to physical or mental illness to
               substantially perform his duties on a full-time basis for six (6)
               consecutive months and, within thirty (30) days after a notice of
               termination is thereafter given by the Company, the Executive
               shall not have returned to the full-time performance of the
               Executive's duties; provided, however, if the Executive shall not
               agree with a determination to terminate him because of
               Disability, the question of the Executive's disability shall be
               subject to the certification of a qualified medical doctor agreed
               to by the Company and the Executive or the Executive's legal
               representative, in the event of the Executive's incapacity to
               designate a doctor. In the absence of agreement between the
               Company and the Executive, each party shall nominate a qualified
               medical doctor and the two doctors shall select a third doctor,
               who shall make the determination as to Disability.


                    (b)  Retirement.  Termination by the Executive or by 
                         ----------
               Certified of the Executive's employment based on "Retirement"
               shall mean termination on or after the Executive's attainment of
               age sixty-five (65) or an earlier age mutually agreed to by the
               Executive and Certified.


                    (c)  Cause.  Termination by Certified of the Executive's 
                         -----    
               employment for "Cause" shall mean termination upon:


                         (i)  the willful and continued failure by the Executive
                              to substantially perform his duties (other than
                              any such failure resulting from the Executive's
                              incapacity due to physical or mental illness),
                              after demand for substantial performance is
                              delivered in writing by the Company to the
                              Executive that 

                                      -2-
<PAGE>
 
                              specifically identifies the manner in which the
                              Company believes the Executive has not
                              substantially performed his duties, or


                         (ii) the willful engaging by the Executive in illegal
                              or fraudulent misconduct which is materially
                              injurious to the Company, or


                        (iii) the willful material breach of the
                              Confidentiality and Nonsolicitation Agreement set
                              forth in Section 6.


               No act, or failure to act, on the Executive's part shall be
               considered "willful" unless done, or omitted to be done, by him
               not in good faith and without reasonable belief that his action
               or omission was in the best interest of the Company.


                    (d)  Good Reason.  Termination by the Executive of the 
                         -----------
               Executive's employment for "Good Reason" shall mean termination
               based on (i) an adverse change in the Executive's status or
               position(s), in effect immediately prior to the date of this
               Agreement, (ii) a reduction in the Executive's base salary as in
               effect immediately prior to the date of this Agreement, (iii) a
               Change in Control, or (iv) the requirement by the Company that
               the Executive be based at an office greater than 35 miles from
               where the Executive's office is located on the date of the Change
               of Control.


                    (e)  Notice of Termination.  Any purported termination by 
                         ---------------------  
               Certified or by the Executive shall be communicated by written
               Notice of Termination to the other party hereto. For purposes of
               this Agreement, a "Notice of Termination" shall mean a notice
               which shall indicate the specific termination provision in this
               Agreement relied upon.


                    (f)  Date of Termination.  "Date of Termination" shall be 
                         -------------------
               the date set forth by written Notice of Termination or, if none,
               then by mutual written agreement of the parties or by the
               arbitrator in a proceeding as provided in Section 11 hereof.
               During the pendency of any such dispute, Certified will continue
               to pay the Executive the Executive's full compensation in effect
               just prior to the time the Notice of Termination is given and
               until the dispute is resolved in accordance with Section 11.

                                      -3-
<PAGE>
 
     4.   COMPENSATION UPON TERMINATION.
          ----------------------------- 


               (a)  Subject to Section 8 hereof, if, the Executive's employment
          with Certified shall be terminated (i) by Certified other than for
          Cause, Disability, Death or Retirement or (ii) by the Executive for
          Good Reason, then within five business days of the Date of
          Termination, Certified shall pay the Executive an amount equal to the
          Executive's highest annual base salary during the three year period
          immediately prior to the Date of Termination, plus an amount equal to
          the average annual incentive bonus paid during the three years prior
          to the Date of Termination.


               (b)  Certified shall maintain all benefits for the Executive and
          his dependents in substantially the same form as those benefits in
          place immediately prior to the Executive's termination for a period
          terminating on the earliest of (i) twelve (12) months from the Date of
          Termination, (ii) the commencement of similar benefits from a new
          employer (but only to the extent that equivalent benefits are offered
          by said employer), or (iii) attainment of age 65.


               (c)  All payments and benefits provided for yet unpaid in this
          Section 4 shall be forfeited if the Executive violates any material
          provision of this Agreement including, without limitation, the
          Confidentiality and Nonsolicitation Agreement specified in Section 6
          herein.


               (d)  In the event of termination of the Executive by the Company
          for Cause, death, Disability, Retirement, or by the Executive for
          other than Good Reason, no payment shall be provided under this
          Agreement. This Agreement does not, and is not intended to, limit any
          rights or benefits of the Executive pursuant to any other plan, policy
          or written agreement.


     5.   NO OBLIGATION TO MITIGATE.
          ------------------------- 


          The Executive is under no obligation to mitigate damages in the amount
          of any payment provided for hereunder by seeking other employment or
          otherwise.


          Subject to paragraph 4(b), the amount of any payment provided for in
          this Agreement shall not be reduced, offset or subject to recovery by
          Certified by reason of any compensation earned by the Executive as the
          result of employment by another employer after the Date of
          Termination, or otherwise.


     6.   CONFIDENTIALITY AND NONSOLICITATION AGREEMENT.
          --------------------------------------------- 


               (a)  The Executive acknowledges that in the course of his
          employment by the Company, he will have access to and become informed
          of confidential and secret information which is a competitive asset of
          the Company ("Confidential Information"), including (i) the terms of
          any agreement between 

                                      -4-
<PAGE>
 
          the Company and any employee, customer or supplier, (ii) pricing
          strategy, (iii) product development strategies, (iv) personnel
          training and development programs, (v) financial results, (vi)
          strategic plans and demographic analyses, (vii) proprietary computer
          and systems software, and (viii) any confidential non-public
          information received from the Company concerning the Company, its
          employees, suppliers and customers. The Executive agrees that he will
          keep all Confidential Information in strict confidence during the term
          of his employment by the Company and thereafter and will never make
          known, divulge, reveal, furnish, make available, or use any
          Confidential Information (except in the course of his regular
          authorized duties on behalf of the Company). The Executive agrees that
          the obligations of confidentiality hereunder shall survive termination
          of his employment at the Company regardless of any actual or alleged
          breach by the Company of this Agreement and shall continue for one
          year following such termination provided that such obligation shall
          terminate earlier (i) as to specific information that shall have
          become known through no fault of the Executive or (ii) as to
          Confidential Information which the Executive is required by law to
          disclose (after giving the Company notice and an opportunity to
          contest such requirement). The Executive's obligations under this
          Section 6 are in addition to, and not in limitation or preemption of,
          all other obligation of confidentiality which the Executive may have
          to the Company under general legal or equitable principles.


               (b)  Except in the ordinary course of the Company's business, the
          Executive has not made, nor shall at any time following the date of
          this Agreement, make or cause to be made, any copies, pictures,
          duplicates, facsimiles, or other reproductions or recordings or any
          abstracts or summaries including or reflecting Confidential
          Information. All such documents and other property furnished to the
          Executive by the Company or otherwise acquired or developed by the
          Company shall at all times be the property of the Company. Upon
          termination of the Executive's employment by the Company, the
          Executive will return to the Company any such documents or other
          property of the Company which are in the possession, custody or
          control of the Executive.


               (c)  In the event of the Executive's termination of employment at
          the Company, the Executive agrees that he will not in any capacity, on
          his own behalf or on behalf of any other firm, person, or entity, for
          a period of one year, solicit, or assist in the solicitation of, any
          employee of the Company to terminate his or her employment with the
          Company.


               (d)  The Executive acknowledges and agrees that a violation of
          the foregoing provisions of this Section 6 (referred to collectively
          as the Confidentiality and Nonsolicitation Agreement) that results in
          material detriment to the Company would cause irreparable harm to the
          Company, and that the Company's remedy at law for any such violation
          would be inadequate. In recognition of the foregoing, the Executive
          agrees that, in addition to any other 

                                      -5-
<PAGE>
 
          relief afforded by law or this Agreement, including damages sustained
          by a breach of this Agreement and forfeiture of any and all
          compensation or benefit otherwise provided under Section 4, and
          without any necessity or proof of actual damages, the Company shall
          have the right to enforce this Agreement by specific remedies, which
          shall include, among other things, temporary and permanent
          injunctions, it being the understanding of the undersigned parties
          hereto that damages, the forfeitures described above and injunctions
          shall all be proper modes of relief and are not to be considered as
          alternative remedies.


     7.   SUCCESSORS; BINDING AGREEMENT.
          ----------------------------- 


               (a)  The Company will require any successor (whether direct or
          indirect, by purchase, merger, consolidation, reorganization or
          otherwise) by agreement in form and substance satisfactory to the
          Executive, expressly to assume and agree to perform this Agreement in
          the same manner and to the same extent the Company would be required
          to perform if no such succession had taken place.


               (b)  This Agreement shall inure to the benefit of and be
          enforceable by the Executive's personal or legal representatives,
          executors, administrators, successors, heirs, distributees, devisees,
          and legatees. If the Executive should die while any amount would still
          be payable to the Executive hereunder if he had continued to live, all
          such amounts, unless otherwise provided herein, shall be paid in
          accordance with the terms of this Agreement to the Executive's
          devisee, legatee, or other designee, or if there be no such designee,
          to the Executive's estate.


               (c)  This Agreement, and all of the provisions hereof, shall be
          binding upon Certified and all of its affiliates, successors,
          transferees, or surviving or continuing entity.


     8.   TAXES.
          ----- 


               (a)  All payments to be made to the Executive under this
          Agreement will be subject to required withholding of federal, state,
          and local income and employment taxes.


               (b)  Notwithstanding anything in the foregoing to the contrary,
          if any of the payments provided for in this Agreement, together with
          any other payments which the Executive have the right to receive from
          Certified, would constitute a "parachute payment" (as defined in
          Section 280G(b)(2) of the Code), the payments pursuant to this
          Agreement shall be reduced to the largest amount as will result in no
          portion of such payments being subject to the excise tax imposed by
          Section 4999 of the code; provided, however, that the determination as
          to whether any reduction in the payments under this Agreement 

                                      -6-
<PAGE>
 
          pursuant to this provision is necessary shall be made by the Executive
          in good faith, and such determination shall be conclusive and binding
          on Certified with respect to its treatment of the payment for tax
          reporting purposes.


     9.   NOTICE.  For purposes of this Agreement, notices and all other
          ------                                                        
          communications provided for in the Agreement shall be in writing and
          shall be deemed to have been duly given when delivered by United
          States registered or certified mail, return receipt requested, postage
          prepaid and addressed, in the case of Certified, to the address set
          forth on the first page of this Agreement or, in the case of the
          undersigned employee, to the address set forth below his signature,
          provided that all notices to Certified shall be directed to the
          attention of the Chief Executive Officer of the Certified, with a copy
          to the Secretary of Certified, or to such other address as either
          party may have furnished to the other in writing in accordance
          herewith, except that notice of change of address shall be effective
          only upon receipt.


     10.  VALIDITY.  The invalidity or unenforceability of any provision of this
          --------                                                              
          Agreement shall not affect the validity or enforceability of any other
          provision of this Agreement, which shall remain in full force and
          effect.


     11.  ARBITRATION.  Any dispute or controversy arising under or in
          -----------                                                 
          connection with this Agreement shall be settled exclusively by an
          arbitrator selected under the rules of the American Arbitration
          Association and the arbitration shall be conducted under the rules of
          said association.  Any claim must be filed within thirty (30) days of
          the Notice of Termination unless extended by mutual written agreement
          by both parties.  Each party shall be entitled to present evidence and
          argument to the arbitrator.  The arbitrator shall have the right only
          to interpret and apply the provisions of this Agreement and may not
          change any of its provisions.  The arbitrator shall permit reasonable
          pre-hearing discovery of facts, to the extent necessary to establish a
          claim or a defense to a claim, subject to supervision by the
          arbitrator.  The hearing shall begin within thirty (30) days of the
          filing of the claim, unless extended by mutual written agreement by
          both parties, and the arbitrator's decision shall be rendered within
          thirty (30) days of the completion of the hearing unless extended by
          mutual written agreement by both parties.  The determination of the
          arbitrator shall be conclusive and binding upon the parties and
          judgment upon the same may be entered in any court having
          jurisdiction.  The Company shall bear all direct costs of the
          arbitration proceedings pursuant to this Section 11 (arbitration fees,
          transcript expenses, etc.) and pay reasonable legal fees of the
          Executive.


     12.  SURVIVAL OF PROVISIONS.  Notwithstanding any other provision of this
          ----------------------                                              
          Agreement, the parties' respective rights and obligations under
          Sections 4, 5, 6, and 7, 11 of this Agreement shall survive
          termination of this Agreement.

                                      -7-
<PAGE>
 
     13.  MISCELLANEOUS.  No provision of this Agreement may be modified, waived
          -------------                                                         
          or discharged unless such waiver, modifications or discharge is agreed
          to in writing signed by the Executive and the Company.  No waiver by
          either party hereto at any time of any breach by the other party
          hereto or compliance with any condition or provision of this Agreement
          to be performed by such other party will be deemed a waiver of similar
          or dissimilar provisions or conditions at the same or at any prior or
          subsequent time.  Unless otherwise noted, references to "Sections" are
          to sections of this Agreement.  The captions used in this Agreement
          are designed for convenient reference only are not to be used for the
          purpose of interpreting any provision of this Agreement.


     14.  ENFORCEABILITY.  Notwithstanding any other provision of this
          --------------                                              
          Agreement, to the extent that any payment to be made pursuant to this
          Agreement is prohibited by applicable federal or state law or
          regulation, or by any action of any federal or state regulatory
          agencies, unless Certified has obtained prior approval for such
          otherwise prohibited payment from the appropriate regulatory
          authority, Certified shall not be obligated to make such payments
          under this Agreement.


     15.  GOVERNING LAW.  The validity, interpretation, construction and
          -------------                                                 
          performance of this Agreement will be governed by and construed in
          accordance with the substantive laws of the State of California,
          without giving effect to the principles of conflict of laws of such
          State.


IN WITNESS WHEREOF, the parties hereto have caused this agreement to be duly
executed as of the date set forth below.

CERTIFIED GROCERS OF CALIFORNIA, LTD.

By: 
   _______________________________________
      Alfred A. Plamann
      President

Agreed to this ninth day of June, 1998.

___________________________________________
Robert M. Ling, Jr.

                                      -8-