<PAGE>

                                                                       EXHIBIT 4
            SECOND AMENDMENT TO AMENDED AND RESTATED LOAN AGREEMENT

   This Second Amendment to Amended and Restated Loan Agreement ("Amendment") is
dated as of the 31st day of January, 1997, by and between PMT SERVICES, INC. 
("Borrower"), a Tennessee corporation, and FIRST UNION NATIONAL BANK OF 
TENNESSEE ("Lender"), a national banking association.

                                  WITNESSETH:

   WHEREAS, Borrower and Lender have previously executed that certain Amended 
and Restated Loan Agreement dated as of May 31, 1995, as amended by that First 
Amendment to Amended and Restated Loan Agreement dated as of July 18, 1996 (as 
amended, the "Loan Agreement") (capitalized terms not otherwise defined herein 
have the meaning assigned in the Loan Agreement); and

   WHEREAS, Borrower and Lender wish to further amend the Loan Agreement in 
certain respects;

   NOW, THEREFORE, for good and valuable consideration, the receipt and 
sufficiency of which are hereby acknowledged, the parties hereto agree as 
follows:

   1.  The Loan Agreement is hereby amended by revising the reference to "three 
months" in the first line of Section 1.29 to read "three, six or nine months, as
elected by Borrower in its notice of borrowing".

   2.  The Loan Agreement is hereby amended by revising it to read in full as 
follows:

   "Note" means that Fourth Amended and Restated Promissory Note made by
   Borrower dated as of January 31, 1997, in the principal amount of $20,000,000
   payable to the order of Lender and all modifications, extensions, renewals,
   amendments and restatements thereof.

   3.  The Loan Agreement is hereby amended by adding a Section 1.40(a) thereto,
reading in full as follows:

   "Permitted Acquisition" means any acquisition by Borrower of a Permitted
   Subsidiary or all or substantially all of the operating assets of any Person,
   or assets which constitute all or substantially all of the

<PAGE>
 
  assets of a division or a separate or separable line of business or operating 
  unit, provided that:

    (i)    Borrower or a Permitted Subsidiary, as the case may be, is the 
  surviving entity in any acquisition involving a merger, reorganization or 
  recapitalization;

    (ii)   the Person or line of business acquired is in the same line of 
  business as that of Borrower;

    (iii)  no Event of Default or Unmatured Default shall exist at the time of
  such acquisition or would result on a pro forma basis after completion of such
  acquisition;

    (iv)   the acquisition must have been approved by the Board of Directors or 
  other similar governing authority of the seller or acquiree;

    (v)    Lender must have issued its prior written consent in the case of any
  acquisition which (x) singly or in the aggregate with other acquisitions from
  the same seller or affiliated sellers has a purchase price (including all
  consideration, however denominated, in connection with the acquisition, and
  valuing contingent consideration on a basis acceptable to Lender) in excess of
  $15,000,000.00, or which (y) taken together with all other acquisitions for
  the same fiscal year, brings the total consideration paid for all such
  acquisitions, determined in the manner set forth in the preceding sentence, to
  any amount in excess of $50,000,000.00; provided, however, that for the
  purpose of this section (y), as long as there are no amounts outstanding under
  the Revolving Credit Loan, consideration paid for acquisitions by the issuance
  of Borrower's own stock shall not be counted in calculating the total
  consideration paid for acquisitions.

  4. The Loan Agreement is hereby amended by changing the value "$25,000" in the
third line of Section 1.42.9 thereof to read "$500,000".

  5. The Loan Agreement is hereby amended by adding a Section 1.42(a) thereto 
reading in full as follows:

                                      -2-

<PAGE>
 
     "Permitted Subsidiary" means any Person (i) of which Borrower directly owns
  at least a majority of the voting equity interests, determined on a fully
  diluted basis, (ii) that is a Borrower under this Agreement or a guarantor of
  the Obligations, as Lender may specify, (iii) that is owned by Borrower in a
  proportion sufficient to allow Borrower to Control the Person, including the
  right to cause the Person to make lawful distributions of income, and the
  financial interest of Borrower therein is freely alienable by Borrower through
  a security interest granted therein or otherwise, (iv) as to which Borrower
  has granted to Lender as additional security for the Obligations, a first
  priority perfected security interest in its stock or other equity interest in
  the Person pursuant to documentation in form and substance acceptable to
  Lender and its counsel, with the validity and perfection of the security
  interest and other matters as Lender may reasonably require confirmed to
  Lender by an opinion of Borrower's outside counsel satisfactory to Lender in
  all respects, and with all expenses related to such documentation (including,
  but not limited to, filing fees and taxes and the reasonable fees and expenses
  of Lender and its attorneys) to be paid by Borrower, and (v) only if so
  requested by Lender, which has granted to Lender a first priority perfected
  security interest in its assets pursuant to documentation in form and
  substance acceptable to Lender and its counsel, with the validity and
  perfection of the security interest and other matters as Lender may reasonably
  require confirmed to Lender by an opinion of Borrower's outside counsel
  satisfactory to Lender in all respects, and with all expenses related to such
  documentation (including, but not limited to, filing fees and taxes and the
  reasonable fees and expenses of Lender and its attorneys) to be paid by
  Borrower.

  6.  Section 2.1 of the Loan Agreement is amended by replacing the words and 
figures "Seventeen Million Five Hundred Thousand and No/100 Dollars 
($17,500,000.00)" with the words and figures "Twenty Million and No/100 Dollars 
($20,000,000.00)."

  7.  The Loan Agreement is hereby amended by revising Sections 2.2.1 and 2.2.2 
thereof to read in full as follows:

                                      -3-

<PAGE>
 
                2.2.1 LIBOR Loans shall bear interest at the rate of one hundred
        twenty-five (125) basis points above the Adjusted LIBOR Rate, which
        shall remain fixed for each LIBOR Interest Period.

                2.2.2 Prime Rate Loans shall bear interest at the rate of one-
        half of one percent (0.50%) below the Prime Rate, as it may change from
        time to time.

        8. Sections 2.4 and 2.5.3 are amended by replacing the date "November 1,
1996" with "January 31, 1998".

        9. The Loan Agreement is hereby amended by deleting the period at the 
end of the third sentence thereof and replacing it with the phrase ", and the 
requested LIBOR Interest Period, if applicable."

        10. The Loan Agreement is hereby amended by revising Section 2.8.1 
thereof to read in full as follows:

                2.8.1 Amounts converted to a term loan shall bear interest, at
        Borrower's election, at either the Prime Rate or Adjusted LIBOR Rate
        option described in Section 2.2 hereof, or at a fixed rate equal to 200
        basis points in excess of the bond equivalent bid side yield of the U.S.
        Treasury Note with a maturity closest to January 31, 2002.

        11. The Loan Agreement is hereby amended by revising Section 2.8.3 to 
read in full as follows:

                2.8.3 All remaining principal and interest shall be due and 
        payable on January 31, 2002.

        12. The Loan Agreement is hereby amended by deleting the text of Section
3.1.9 and replacing it with the phrase "[intentionally left blank]," it being 
the intention by this deletion that the life insurance requirement set forth 
therein shall no longer apply. Accordingly, the life insurance presently held by
Lender to secure the Obligations shall be released.

        13. The Loan Agreement is hereby amended by adding the following as a 
new final sentence in Section 5.2.5 thereof:

        Without limiting the foregoing, Borrower shall deliver its 10K prepared
        pursuant to applicable securities laws within ninety (90) days after the
        end of each fiscal year

                                      -4-
<PAGE>
 
        and shall deliver its 10Q, similarly prepared, within forty-five (45) 
        days after the end of each fiscal quarter.

        14. The Loan Agreement is hereby amended by revising Section 6.1 thereof
to include the words "shall not" before the word "exceed" in the first line 
thereof.

        15. The Loan Agreement is hereby amended by revising the ratio required 
in Section 6.2 thereof from 2.00:1.00 to 1.75:1.00.

        16. The Loan Agreement is hereby amended by deleting the text of Section
6.4 thereof and substituting the phrase "[intentionally left blank]", it being 
the intention that the net income requirement no longer apply.

        17. The Loan Agreement is hereby amended by revising the text of Section
7.1 to include additional Sections 7.1.7 and 7.1.8 providing in full as follows:

        7.1.7 CERTAIN GUARANTIES. Guaranty obligations (i) in existence as of
March   , 1997, as disclosed to Lender in a written schedule dated as of that
date, (ii) with respect to Bankcard America in an amount of up to $1,000,000,
(iii) with respect to Money Transfer Systems in an amount of up to $250,000, 
and (iv) other guaranty obligations incurred in the ordinary course of business 
in favor of processing banks with regard to processing operations or in favor of
Sirrom Capital Corporation or its Affiliates with regard to other payment 
servicing companies, subject to the limitations that no guaranty obligation 
shall be permitted under this Subsection 7.1.7(iv) in excess of the amount of 
$1,000,000 as to any individual primary obligor and the aggregate guaranty 
obligations permitted under this Subsection 7.1.7(iv) shall not exceed 
$25,000,000.

        7.1.7 ADDITIONAL GUARANTIES. Additional guaranty obligations incurred in
the ordinary course of business in an amount not exceeding $5,000,000 in the 
aggregate.

        18. The Loan Agreement is hereby amended by revising the text of Section
7.2 thereof to provide that only Richardson M. Roberts and Gregory S. Daily are 
covered by the restriction on change in management.

        19. The Loan Agreement is hereby amended by deleting the text of Section
7.3 thereof and substituting the phrase "[intentionally left blank]", it being 
the intention that the capital expenditure limit no longer apply.

                                      -5-






            
<PAGE>
 
        20. The Loan Agreement is hereby amended by revising each of Sections 
7.5 and 7.8 by deleting the final periods thereof and adding the phrase 
", except for Permitted Acquisitions."

        21. Concurrently with the execution hereof, Borrower shall pay Lender a 
$25,000 commitment fee, which fee is neither refundable nor proratable. Borrower
shall also pay all recording costs, attorneys' fees and other fees and expenses 
incurred in connection with this Amendment.

        22. Borrower warrants to Lender that, after giving effect to this
Amendment, no Event of Default or Unmatured Event of Default exists under the
Loan Agreement, except as follows. Borrower is a party to certain litigation
that needs to be reported to Lender and has guaranteed certain obligations for
which Lender's consent is required. On or before March 31, 1997, Borrower shall
provide Lender with information sufficient to enable Lender to assess these
matters and to issue any necessary consents, should Lender so elect.
Additionally, as of the execution of this Amendment, Borrower has several
Subsidiaries that are not "Permitted Subsidiaries" under the definition provided
above in this Amendment. Lender hereby waives noncompliance with the above
requirements as to Subsidiaries until the funding of any advance under the
Revolving Credit Loan.

        23. Prior to and as a condition to the funding of any advance under the 
Revolving Credit Loan, Borrower shall cause Lender to receive (i) the disclosure
items referred to in Section 22 above, (ii) stock pledges, guaranties, corporate
charters, resolutions and such other documents as Lender may require in 
connection with the qualification of Subsidiaries as "Permitted Subsidiaries," 
and (iii) such other amendments, resolutions of Borrower's Board of Directors, 
opinions of Borrower's counsel and other documents as Lender or its counsel may 
reasonably require to evidence and secure the Obligations consistent with the 
requirements of the Loan Agreement as amended hereby. In order to allow time for
the preparation and execution of such documentation, Borrower agrees to submit 
its initial borrowing request under the Loan Agreement at least thirty (30) days
prior to the requested disbursement date.

        24. The provisions hereof shall control provisions of the Loan Agreement
to the extent the two documents may conflict. The validity and construction 
hereof shall be determined in accordance with the substantive laws of the State 
of Tennessee.

                                      -6-

<PAGE>
 
Executed as of the date first written above.


                                  PMT SERVICES, INC.

 
                                  By: /s/ Clay Whitson
                                     -------------------------------

                                  Title: Chief Financial Officer
                                        ----------------------------


                                  FIRST UNION NATIONAL BANK OF 
                                  TENNESSEE
                                 
                                
                                  By: /s/ Orville W. Kronk
                                     -------------------------------

                                  Title: Vice President
                                        -----------------------------

                                      -7-