<PAGE>   1

                                                                Exhibit 10.6
                                  May 31, 2000



Jonathan Block, Esq.
General Counsel
Salem Communications Corp.
4880 Santa Rosa Road
Suite 300
Camarillo, California  93012

                  RE:      ASSET EXCHANGE AGREEMENT BY AND AMONG COX RADIO,
                           INC., CXR HOLDINGS, INC., SALEM COMMUNICATIONS
                           CORPORATION AND SOUTH TEXAS BROADCASTING, INC., DATED
                           AS OF MAY 31, 2000 (THE "ASSET EXCHANGE AGREEMENT")

Dear Mr. Block:

         This letter is provided in connection with the above-referenced Asset
Exchange Agreement. Unless otherwise defined herein, all capitalized terms used
herein shall have the same meaning ascribed to such terms in the Asset Exchange
Agreement.

         Pursuant to the exercise of a right of first refusal granted to CRI
under an Agreement for Right of First Refusal dated as of April 3, 1995 between
WSB Radio, Inc., predecessor-in-interest to CRI, and Ring Radio Company ("RRC"),
on June 1, 2000, Cox will enforce diligently the right of first refusal and take
all steps reasonably necessary to enforce its rights thereunder including,
without limitation, the execution and delivery of a Stock Purchase Agreement
(the "Stock Purchase Agreement") among CRI, Midwestern Broadcasting Company,
Inc. ("Midwestern"), and the stockholders of Midwestern (the "Stockholders")
pursuant to which CRI will agree to acquire all of the issued and outstanding
capital stock of Midwestern. Midwestern is the sole stockholder of RRC.

         a.       Stock Purchase Agreement. Section 8.7 of the Stock Purchase
Agreement provides that CRI shall not be obligated to close the transactions
under the Stock Purchase Agreement unless the Stockholders shall have terminated
the Station Affiliation Agreement (as defined in the Stock Purchase Agreement)
with OURBUS without any continuing liability or obligations on the part of RRC
or Midwestern under such agreement. Notwithstanding the terms of Section 8.7 of
the Stock Purchase Agreement or any provision in the Asset Exchange Agreement to
the contrary, CRI and Salem hereby agree that upon Salem's reasonable request,
CRI will waive the Stockholders' compliance with the requirements of Section 8.7
of the Stock Purchase Agreement.


<PAGE>   2

Jonathan Block, Esq.
May 31, 2000
Page 2


         b.       Press Release. Salem and Cox agree that neither party
shall issue a press release concerning the transactions contemplated by the
Stock Purchase Agreement or the Asset Exchange Agreement prior to 8:00 a.m.,
Eastern Daylight Time, on Friday, June 2, 2000.

         c.       Programming Agreements. Salem and Cox agree that with respect
to the Salem Station, no later than the date on which the FCC Applications are
filed, Salem shall provide notice of termination to the appropriate third
parties under those agreements for programming on the Salem Station's affiliate
station, KENR(AM), that require prior written notice of termination. In the
event that the Closing occurs prior to the expiration of said notice period, Cox
and Salem shall reasonably cooperate to complete the airing of such programming
on the Salem Station. After the Closing, Cox and Salem also shall reasonably
cooperate to transition the Salem Station's programming to KENR(AM), which, at
this time, Salem anticipates will take no less than two weeks.

         d.       FCC Licenses. Contemporaneously with the execution of the
Asset Exchange Agreement, Salem learned that the antenna monitors for the Cox
Stations may have been replaced. In connection therewith, and to the extent
required by the FCC's rules and regulations, Cox shall conduct a proof of
performance with respect to the Cox Stations prior to the Closing.

         e.       Environmental Assessment. Within ten (10) business days after
the execution of the Asset Exchange Agreement, Cox shall provide Salem and Salem
shall provide Cox with the originals or readable copies of any environmental
assessments with respect to such party's Stations that are in such party's
possession or control. Prior to Closing, Cox may obtain, at its option and
expense, an assessment of the Salem Real Property by an environmental engineer
selected by Cox, and Salem may obtain, at its option and expense, an assessment
of the Cox Real Property and the RRC Real Property by an engineer selected by
Salem (in either case, the "Environmental Assessment"). Each Environmental
Assessment shall be subject to the confidentiality provisions of the Asset
Exchange Agreement. If, after appropriate inquiry into the previous ownership of
and uses of the Cox Real Property, the RRC Real Property or the Salem Real
Property, as the case may be, consistent with good commercial or customary
practice, a party's engineer concludes, as set forth in the Environmental
Assessment, that environmental conditions exist on, under or affecting such
properties that would constitute a material violation or breach of the conveying
party's representations and warranties contained in the Asset Exchange Agreement
then notwithstanding any other provisions of the Asset Exchange Agreement to the
contrary, but subject to the following sentence, the party conveying such real
property shall at its sole cost and expense (up to a maximum amount of Fifty
Thousand Dollars ($50,000)) remove, correct or remedy any condition or
conditions which constitute a material violation or breach of such party's
representations and warranties prior to the Closing Date and provide to the
acquiring party at Closing a certificate from an environmental abatement firm
that such removal, correction or remedy has been completed so that the party's
representations and warranties with respect to environmental matters will be
true and correct in all material respects as of the Closing Date. In the event
the cost of removal, correction or remedy of the environmental conditions
exceeds Fifty Thousand Dollars ($50,000), the acquiring party may elect to
proceed with the Closing but shall not be obligated to close under any
circumstances which would require the acquiring party to assume ownership of a
Station under conditions


<PAGE>   3

Jonathan Block, Esq.
May 31, 2000
Page 3


where there exist any material uncured violations of the conveying party's
warranties, representations or covenants with respect to environmental matters.

         Please indicate your agreement to the foregoing by signing in the space
provided below.

         This letter agreement may be executed in multiple counterparts, each of
which shall be deemed an original and all of which taken together shall
constitute one and the same instrument.

                        [SIGNATURES FOLLOW ON NEXT PAGE]


<PAGE>   4

Jonathan Block, Esq.
May 31, 2000
Page 4


                                    Very truly yours,

                                    COX RADIO, INC.



                                    By:
                                        -----------------------------------
                                             Maritza C. Pichon
                                             Chief Financial Officer


                                    CXR HOLDINGS, INC.



                                    By:
                                        -----------------------------------
                                             Richard F. Klumpp
                                             Assistant Secretary

ACCEPTED AND AGREED:

SALEM COMMUNICATIONS CORPORATION
SOUTH TEXAS BROADCASTING, INC.



By:
    -----------------------------
    Name:
    Title:


cc: James P. Riley, Esq.
    Kevin F. Reed, Esq.