<PAGE>
 
                      SECURITIES AND EXCHANGE COMMISSION
                            WASHINGTON, D.C.  20549
                                  FORM 10-K/A
                                AMENDMENT NO. 1

                                        
(MARK ONE)

  {X}       ANNUAL REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE
            SECURITIES EXCHANGE ACT OF 1934
            For the fiscal year ended            December 31, 1997         .
                                      --------------------------------------
 
  { }       TRANSITION REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE
            SECURITIES EXCHANGE ACT OF 1934
            For the transition period from ____________ to _____________.
                                    0-20364
                                    -------
                           (Commission file number)


                          BANYAN SYSTEMS INCORPORATED
            (Exact name of registrant as specified in its charter)

               MASSACHUSETTS                         04-2798394
     (State or other jurisdiction of     (I.R.S. Employer Identification No.)
      incorporation or organization)

               120 FLANDERS ROAD, WESTBORO, MASSACHUSETTS  01581
             (Address and zip code of principal executive offices)
                                 508-898-1000
                                 ------------
             (Registrant's telephone number, including area code)
       Securities registered pursuant to Section 12(b) of the Act: None
          Securities registered pursuant to Section 12(g) of the Act:
                         Common Stock, $.01 par value
                         ----------------------------
                               (Title of Class)
                               ----------------
 
Indicate by check mark whether the registrant: (1) has filed all reports
required to be filed by Section 13 or 15 (d) of the Securities Exchange Act of
1934 during the preceding 12 months (or for such shorter period that the
registrant was required to file such reports), and (2) has been subject to such
filing requirements for the past 90 days.
Yes   X        No 
     ---          ---    

Indicate by check mark if disclosure of delinquent filers pursuant to Item 405
of Regulation S-K is not contained herein, and will not be contained, to the
best of registrant's knowledge, in definitive proxy or information statements
incorporated by reference in Part III of this Form 10-K or any amendment to this
Form 10-K. [ ]

The aggregate market value of the voting stock held by non-affiliates of the
registrant, based on a per share fair market value of $5.688, was approximately
$90,829,288.73 as of March 23, 1998.  For this purpose, any officer, director or
5% of the Company is deemed to be an affiliate.

On March 23, 1998, there were 17,942,860 shares of Common Stock and 263,158
shares of Series A Convertible Preferred Stock outstanding.


                      DOCUMENTS INCORPORATED BY REFERENCE

(1) Specifically identified portions of the Annual Report to Shareholders for
    the fiscal year ended December 31, 1997 (the "Annual Report to
    Shareholders") are incorporated by reference into Parts I and II.

(2) Specifically identified portions of the Company's Definitive Proxy Statement
    to be filed in connection with the Company's 1998 annual meeting of
    stockholders are incorporated by reference into Part III.

                                      -1-
<PAGE>
 
     Exhibits 13 and 23 to the Annual Report on Form 10-K of Banyan Systems 
Incorporated for the fiscal year ended December 31, 1997, as filed with the 
Securities and Exchange Commission on March 31, 1998 (the "Banyan 1997 Form 
10-K"), are hereby amended and restated in their entirety as filed herewith, and
Item 1 of Part I, Item 8 of Part II and Item 14 of Part IV to the Banyan 1997 
Form 10-K are hereby amended and restated in their entirety as follows:

PART I

ITEM 1.   BUSINESS

GENERAL

Banyan Systems Incorporated ("Banyan" or the "Company") designs, develops and
markets standards-based networking directory and messaging products that help
people to communicate across enterprise networks, intranets and the Internet.
Since 1985, the Company has been a pioneer in the computer networking field, and
currently offers a wide range of software products and professional services.
The Company's primary software products include StreetTalk for Windows NT(R),
VINES(R), Intelligent Messaging(TM), BeyondMail(TM), and several intranet
solutions. In addition, the Company delivers professional services including
technical support, education and consulting, including network performance,
integration and Year 2000 compliance services. The Company's majority-owned
subsidiary, Switchboard Incorporated, hosts the Internet's leading people-to-
people and business directory services aimed at generating advertising revenues
from major domestic corporations. Banyan's products fall into two major
categories: network operating system products and intranet products.

The Company's flagship network operating system product is based on a global
directory service called StreetTalk(TM).  This service allows users to find one
another and access network resources, such as printers, files, mail and
applications, regardless of physical location. Banyan's other enterprise
directory services include messaging, management and security. Together, these
services make up the foundation for Banyan's network operating system products.
These products integrate mainframes, minicomputers, workgroup networks and
personal computers into a unified computing environment that is easy for users
to navigate and administrators to  manage.

--------------------------------------------------------------------------------
Banyan, the Banyan logo, StreetTalk, VINES and BeyondMail are registered
trademarks, and Intelligent Messaging, Switchboard Banyan  Enterprise Network
Solutions and EBR are trademarks of Banyan Systems Incorporated.  StreetTalk is
a product of Banyan Systems Incorporated and not a product of McCarthy, Crisanti
& Maffei, Inc.  UNIX is a registered trademark of  Novell, Inc. in the United
States and in other countries licensed exclusively  through X/Open Company Ltd.
IBM and OS/2 are registered trademarks of  International Business Machines
Corporation.  Microsoft, Windows NT and Windows  are registered trademarks of
Microsoft Corporation. Macintosh is a registered trademark of Apple Computer,
Inc. Other product, company or organization names cited in this Form 10-K are
trademarks or registered trademarks of their respective companies or
organizations.
--------------------------------------------------------------------------------

                                      -2-
<PAGE>
 
The Company delivers its network operating system products in several versions,
including StreetTalk for Windows NT(R) and VINES(R). Introduced in September
1996 and enhanced throughout 1997, Banyan's StreetTalk for Windows NT provides
Banyan's premier enterprise directory capabilities running natively on 
Microsoft(R) Windows NT, an emerging standard for corporate networks. VINES is
Banyan's well proven, UNIX(R) based networking software, which combines file and
print sharing with the StreetTalk directory, messaging, network management and
security services. The Company's Intelligent Messaging(R) option, available for
both VINES and Windows NT environments, provides an enterprise solution for
reliably storing and transporting electronic messages.

Banyan's intranet products enable its customers to utilize Internet standards to
allow people to contact other people and information in a reliable and secure
manner over the Internet.  The Intranet Access product brings all the
capabilities of the network operating system to a user of a browser on the
Internet.  Banyan's Intranet Protect product helps customers authenticate users
and authorize the user access to web site pages.  These are the first products
of Banyan's intranet product initiative.

The Company's service organization includes technical response centers, 
education services and consulting. A critical element of the Company's strategy 
is to assure that customers are provided with technical support, training and 
consulting services required to build, manage and optimize the network 
environment using Banyan products. Consulting services are generally offered as 
customized consulting services and in certain cases as packaged services. When 
combined with other third party product and services, they are positioned as 
solution offerings. The Company provides services through regional offices 
located in the Americas, Europe, Asia Pacific and Australia.

In February 1996, the Company launched Switchboard(TM) an Internet directory
service that allows people to locate residential and business listings. To
realize the potential of Switchboard, as well as to fund its future growth, the
Company established Switchboard Incorporated as a Banyan subsidiary.
Subsequently, in November 1996, the Company entered into an agreement with
America Online, Inc. (AOL) and Digital City Inc. (DCI) to sell minority
interests in Switchboard Incorporated to AOL and DCI. At December 31, 1997,
these companies held in the aggregate approximately a 10% equity interest in
Switchboard Incorporated. See Note P to the Company's Consolidated Financial
Statements, which is incorporated herein by reference to the Company's Annual
Report to Shareholders.

In the second quarter of 1997, the Company announced a reorganization of its
operations.  As a result of the reorganization, the Company recorded net pre-tax
restructuring and other charges in 1997 of $8.0 million. The restructuring and
other charges are expected to reduce cash flow by approximately $3.1 million, of
which $2.7 million had been expended through December 31, 1997.  During 1997,
the restructuring resulted in 78 employee separations.

In the third quarter of 1997, the Company entered into a $15.0 million line of
credit agreement (the "Credit Agreement") with Foothill Capital Corporation
("Foothill"), replacing the Company's prior $10.0 million credit facility which
expired in May 1997. In general, the Company's obligations under the Credit
Agreement bear interest at the variable base rate per annum of Norwest Bank
Minnesota, National Association.  The Credit Agreement has a three-year initial
term and is renewable thereafter for successive one year periods.  Each year
during the initial term of the Credit Agreement, Foothill will be granted
warrants to purchase 75,000 shares of the Company's common stock at the then
current fair market value.  There were no amounts outstanding under the line of
credit agreements during the years ended December 31, 1997 and 1996.

On March 6, 1998, the Company received a $10.0 million equity investment from
HarbourVest Partners LLC. Under the terms of the purchase agreement, HarbourVest
purchased convertible preferred stock and warrants to acquire additional
convertible preferred stock. In the event the Company's stockholders do not
approve an increase in the number of shares of common stock authorized for
issuance from 25,000,000 to 35,000,000 by June 30, 1998, the Company may be
required to repurchase 40% of the HarbourVest preferred stock and warrants for
$4.0 million. On March 6, 1998, the Company received $10.0 million in proceeds
from the sale of preferred stock which are available for working capital
requirements.

                                      -3-
<PAGE>
 
BANYAN ARCHITECTURE AND TECHNOLOGY

The Company bases its software products on a modular architecture, specifically
designed to serve computing environments with  multiple servers in multiple
locations.  This architecture enables the delivery of a set of sophisticated
network services that integrate workgroup LANs (Local Area Networks), the
Internet and other proprietary computer environments into a single enterprise
network.

Banyan's enterprise directory services consist of software modules, each
providing a different service that may be shared by users across the network.
At the core of these services is Banyan's StreetTalk directory.  StreetTalk
enables network administrators to assign location-independent "StreetTalk" IDs
or names to each user, computer, printer, server or other resource in the
network.  These names remain consistent throughout the network, allowing a user
to access any network resource without regard to the physical location of the
user or the resource.  The name, network location and characteristics of each
resource are stored in the StreetTalk database.  StreetTalk's database is used
by all of the other services and integrated applications to locate and
transparently access resources throughout the network.

Banyan has expanded this architecture to include the capability to easily access
these services through the Internet.  From a browser, anywhere on the Internet,
a user can gain access to their printer, file and mail services.  In addition,
for web servers in a corporate intranet, this architecture provides the means to
authenticate users and protect web pages.

All of Banyan's enterprise directory services are integrated with the StreetTalk
directory, including messaging, management, security, local and wide-area
communications, and host connectivity.  As a result of this modular
architecture, integration of services throughout the network is easily
maintained even as individual services are enhanced or additional services are
added.

Banyan's integrated enterprise directory services create the "look and feel" of
a single system for users of a complex network of heterogeneous personal
computers, servers, minicomputers and mainframes.  For example, users log on to
the network rather than individual servers, and thereby gain access to all of
the resources on the network rather than the resources available on an
individual server.  Banyan believes its products supply users with a unified
view of the network, thereby shielding them from its underlying complexities.
Banyan believes its products provide network administrators with the ease-of-
access and centralized management capabilities previously associated only with
minicomputers and mainframe systems.

Banyan believes that its software architecture has enabled it to develop
networking products that offer advantages over other networking products that
have been designed to connect PCs within a workgroup LAN for the purpose of file
and print sharing. Through its integrated set of enterprise directory services,
Banyan seeks to meet the needs of the enterprise networking market with products
that:

  - Provide consistency across the entire network in managing, administrating,
    locating and controlling access to users, resources, applications and data;

  - Permit the network to be easily reconfigured and expanded to include
    additional users, computers, workgroup LANs and sites;

  - Ensure the integrity and security of information distributed throughout the
    network;

  - Provide for integration with minicomputers and mainframes in the
    organization; and

  - Facilitate the development of client-server applications that support the
    enterprise.

                                      -4-
<PAGE>
 
The Company also believes that it was the first company to offer global
directory services and that it continues to offer among the most comprehensive
and technically advanced set of enterprise directory services in the industry.

Banyan is upgrading its product architecture to comply with Year 2000 standards.
Product changes are being made and delivery to customers is expected to occur
beginning in the first half of 1998.  Failure to deliver these product changes
in a timely manner could have a material adverse effect on the Company.

WINDOWS NT INTEGRATION

With the emergence of Microsoft Corporation's Windows NT operating system, the
Company has released products that allow corporations to integrate Windows NT
into their existing corporate networks. In March 1996, the Company introduced
StreetTalk Access(TM) for Windows NT File and Print software, giving customers a
smooth growth path for deploying Windows NT.  

                                      -5-
<PAGE>
 
The capabilities of the StreetTalk Access product are now included among the
features of StreetTalk for Windows NT. Running natively in Windows NT
environments, StreetTalk for Windows NT is designed to integrate Microsoft's
Windows NT operating system into their existing corporate networks. In June
1997, the Company announced the availability of StreetTalk for Windows NT 7.5,
which offers improved support for TCP/IP standards, new management tools,
including additional support for remote management, speedier disaster recovery,
and a new version of Banyan's integrated messaging services. Additionally, the
Company offers Intelligent Messaging for Windows NT, which delivers highly
scalable enterprise message storage and transport capabilities, providing a
secure foundation for organizational communications.

KEY NETWORK OPERATING SYSTEM PRODUCTS

The Company's key network operating system products include StreetTalk for
Windows NT, VINES, Intelligent  Messaging and BeyondMail.  In addition, the
Company markets a broad range of complementary products and options for these
base products.

STREETTALK FOR WINDOWS NT

StreetTalk for Windows NT provides directory capabilities that extend beyond
individual server communities - encompassing every item and activity in the
enterprise.  This simplifies access to all network resources, regardless of
location.  The combination of Windows NT and StreetTalk for Windows NT supplies
an enterprise-wide application platform and networking solution that scales to
any size, including tens of thousands of users.  StreetTalk also includes
numerous security features.

VINES

VINES integrates Banyan's StreetTalk directory and a full suite of enterprise
network services such as security and management into one package.  VINES
installs on PC servers and clients and permits users of the leading PC desktop
systems - DOS, OS/2(R), Windows(R)95, Windows NT and Macintosh(R) -to share
information and computing resources with each other and with host computing
environments throughout the enterprise.  In December 1997, the Company
introduced VINES 8.0, which offers improved performance and features.

BEYONDMAIL

A full-featured e-mail product, the Company believes BeyondMail is among the
most advanced software available for managing electronic mail. With its unique
rules-based agent technology, BeyondMail helps people manage information with e-
mail, classifying messages intuitively in folders for instant recall and
filtering out junk mail.  When BeyondMail's intelligent agents are combined with
easy-to-customize forms, businesses are able to automate complex procedures,
such as producing database reports and routing purchase orders for approval.
BeyondMail supports Windows clients and takes full advantage of StreetTalk and
Intelligent Messaging capabilities. In August 1997, the Company introduced
BeyondMail 3.0 for Intelligent Messaging, which offers improved performance and
features including enhanced mobile support.

INTELLIGENT MESSAGING

Intelligent Messaging is an enterprise-wide, client/server-based solution for
storing and transporting messages. Tightly integrated with StreetTalk directory
services, Intelligent Messaging eases use, reduces costs and facilitates the
deployment of large-scale messaging networks.

INTERNET PRODUCTS AND SERVICES

As a result of the Company's focus on Internet technology development, in 1997,
the Company released products for Internet users built on the Company's
traditional strengths in directory and messaging technology.  The Internet
versions of BeyondMail offer full-featured messaging capabilities and time-
saving productivity enhancements.

In addition to its traditional, directory-based networking and messaging
solutions, the Company is developing a suite of intranet offerings that enable
organizations to create corporate intranets which broaden the scope of their
networks through the Internet.  In the second quarter of 1997, the Company
introduced Banyan Intranet Connect, the first of the Company's new intranet
product initiatives, intranet server software that provides a flexible and cost-
effective way to access network files, printers and e-mail, using a standard web
browser.  In the fourth quarter of 1997, the Company delivered Banyan Intranet
Protect, a web server solution that enables organizations to protect and share
intranet information.

                                      -6-
<PAGE>
 
YEAR 2000 COMPLIANCE

In 1997, Banyan made investments in upgrading its products to be Year 2000
compliant.  All the Company's current products are scheduled to be upgraded by
the end of 1998.  In addition, the Company will offer customer services to
assist in the planning and implementing organizations' Year 2000 network
upgrades.

OTHER PRODUCTS AND SERVICES

UPGRADES AND SUBSCRIPTIONS

Banyan offers upgrade and subscription programs that enable its customers to
migrate cost-effectively to enhanced versions of StreetTalk for Windows NT,
VINES, BeyondMail and other product offerings as they become available.
Customers are encouraged to purchase, for a fixed annual fee, a subscription
contract for their installed Banyan products.

NETWORK APPLICATIONS

Banyan sells and supports several network applications programs that are
integrated with its enterprise network services.  Electronic mail applications
for Intelligent Messaging are available for personal computers running DOS, OS/2
and the Macintosh operating system. A GUI-based management tool called
StreetTalk Explorer is available for StreetTalk for Windows NT and VINES
products.  MNET, which is sold with Banyan's network management service, is a
utility program that monitors the network in real-time and allows for review of
network-wide operational and performance characteristics from any networked PC,
workstation or server.  The VINES Assistant suite of utilities automates routine
tasks of network administration, such as renaming a user or changing security
privileges for a group of users or services.

THIRD-PARTY APPLICATION SOFTWARE

The Company actively encourages independent software vendors, system
integrators, system vendors and VARs to develop and market application software
for use with StreetTalk, VINES  and BeyondMail. Banyan provides developers with
development tools, including documented APIs for enterprise network services, to
assist them in producing programs which are integrated with Banyan's enterprise
networking products.  In addition, to aid these companies in marketing their
products to Banyan's customer base, the Company publishes an application
directory which includes more than 5,000 software programs that run in Banyan
environments.  Through support for emerging industry standards, the Company
believes that it will greatly expand the number of applications available to its
customers.

SERVICES

A key element of the Company's strategy is to assure that customers are provided
with the technical support, training and consulting services required to build,
manage and optimize the network environment using Banyan products.

The Company maintains technical response centers in Westboro, Massachusetts;
Crawley, England; Tokyo, Japan; and Sydney, Australia.  These centers provide
technical  telephone support, on-line access and, to a lesser extent, on-site
support.  In  addition, pre-sales and post-sales support for partners and end
users is  provided through field systems consultants.

The Professional Services organization includes the consulting services function
and the education services business.  Consulting is a growing segment of        
Banyan's business that offers customers and partners services for network       
optimization, transition and implementation.  These services are generally      
offered as customized consulting services and in certain cases as packaged      
services.  When combined with other third party product and services, they are  
positioned as solution offerings.                    

The Company offers a wide range of training programs for partners and end users.
Training is offered at the Westboro headquarters, as well as various regional
sites.  On-site training classes are also offered at partner's sites as well as
end user locations. In addition, there are 56  worldwide Banyan Certified
Education Centers, which conduct training classes for  users and administrators
of Banyan products.

CUSTOMERS

The Company estimates that it has approximately 7,000 customers with more than
8.0 million users worldwide. Banyan's customers, which include nearly one-half
of the Fortune 500 companies, typically are medium to large-size businesses,
financial institutions, professional organizations, universities, government
entities and not-for-profit organizations with multiple sites dispersed over
wide geographic areas.

For the year ended 1997, Ingram Micro Inc. accounted for approximately 11% of
the Company's revenues.  For the years ended 1996 and 1995, Inacom Corporation
accounted for approximately 11% and 16% of the Company's revenues respectively.
In 1997, 1996 or 1995, no other customer or reseller accounted for  more than
10% of the Company's revenues.  The loss of any significant customer could have
a material adverse effect on the Company.

SWITCHBOARD INCORPORATED

The Company's majority-owned Switchboard subsidiary, the exclusive directory for
America Online's 11 million customers, reinforced its position as the Internet's
top-rated directory in 1997.  Switchboard is an Internet-based directory service
that allows any individual with a web browser to look up the names and street
addresses of more than 106 million individuals and 11 million businesses in the
United States.  This product is available at no charge to the user.  Switchboard
generates revenues through display advertisements that are integrated with its
business listings, categorized to match a user's search criteria  similar to
traditional Yellow Pages telephone directories.

During the year, Switchboard enhanced its functionality, traffic and advertising
base.  To improve the Switchboard site's appeal, Switchboard added a unique web
site search capability called SideClick, software for creating customized
advertisements, and a suite of free software programs that includes e-mail and
web site design tools. Switchboard became the Internet's 10th most frequently
web site in January 1998, according to Media Metrix, an increase from #23 at the
end of 1996.

Today, leading businesses such as Sprint, IBM, Barnes & Noble, and Microsoft(R)
market their products and services on Switchboard.  To build traffic in 1998,
Switchboard established a strategic relationship in late 1997 to attract users
of the popular AltaVista(TM) search engine to the Switchboard site.

                                      -7-
<PAGE>
 
MARKETING, SALES AND DISTRIBUTION

The Company's marketing and distribution strategy focuses most of the Company's
field staff on  direct consultative sales to key decision makers to create
product awareness and achieve design wins.  Demand fulfillment is primarily
accomplished through Banyan's remarketing partners.  The Company's remarketing
partners include aggregators (master value-added resellers that typically resell
to other resellers), system integrators (organizations that integrate third
party products into complete systems for sale to customers) and systems vendors
(hardware manufacturers that integrate third party products into their own
systems for sale to customers) that are qualified to install and support the
Company's products as part of a complete network solution for the customer.
Sales of products through remarketing partners accounted for approximately 60%,
80% and 90% of the Company's revenues in 1997, 1996 and 1995, respectively,
which when combined with certain services and Switchboard sales has increased
the share of the Company's direct business over each of the last two years.  The
Company also sells products directly through its own sales force to certain
large end-user customers.

The Company's remarketing partners in the Americas as of December 31, 1997
include three aggregators and 865 channel partner sales sites.  Banyan has
established authorization levels (Premier Network Integrator and Authorized
Network  Integrator, as well as authorization levels for messaging resellers) to
distinguish each reseller organization's expertise in working with the Company's
products, as well as their overall network design, certified support centers,
training, and global support.  The Company's aggregators are InaCom Corporation,
Ingram Micro and Information Access Inc.

In February 1997, the Company announced a new user-based pricing program, which
is designed to provide greater flexibility for platform deployment and a
simplified purchasing process.  This user-based licensing provides an
alternative to Banyan's previous server-based licensing, allowing customers to
choose the most appropriate pricing model.

The Company's international sales currently are made primarily through
distributors in 75 countries.  These distributors provide products, technical
support and follow-on service to local resellers, which in turn serve end-user
customers.  The Company estimates that its resellers had approximately 300
international sales sites as of December 31, 1997. International sales
accounted for 29%, 24% and 23% of the Company's revenues in 1997, 1996 and 1995,
respectively.  See Note O to the Company's Consolidated Financial Statements,
which are incorporated herein by reference to the Company's Annual Report to
Shareholders, for information as to the Company's revenues attributable to each
of the Company's geographic areas during the past three fiscal years.

During 1997, the Company's international distribution activities continued to be
augmented by a joint venture in Japan as well as two OEM agreements.   During
1995, the Company had entered into a joint venture with Marubeni Corporation and
Nippon Telegraph and Telephone.  At December 31, 1997, these Japanese companies
held a 33% equity interest in Nippon Banyan Systems Kabushiki Kaisha (NBSKK), a
majority-owned subsidiary of the Company. See Note P to the Company's
Consolidated Financial  Statements, which are incorporated herein by reference
to the Company's Annual Report to Shareholders.  In addition, the Company has
OEM relationships with  Hitachi and Siemens Nixdorf, which includes development
and support.

The Company actively supports its resellers and other partners with its own
experienced sales and marketing organization.  The Company's sales staff
solicits prospective customers, provides technical advice with respect to
Banyan's products and works closely with the particular Banyan reseller through
which the customer desires to purchase its networking products.  The Company
believes that the active participation of its sales staff in the selling
process, in conjunction with the efforts of its resellers, is necessary in order
to provide customers with the level of support required for the implementation
of enterprise networks.  Banyan conducts its sales and marketing 

                                      -8-
<PAGE>
 
activities from its principal offices in Westboro, Massachusetts as well as 26
other North American sales offices and 10 international offices located in
Australia, Belgium, France, Germany, Hong Kong, Japan, Malaysia, The Netherlands
and the United Kingdom. As of December 31, 1997, the Company's sales and
marketing organization consisted of 182 employees, of which 123 were in the
North American group and 59 were in the international group.

The Company also has entered into sales and support alliances with various
system vendors, including Wang, Hewlett Packard, NCR and Digital Equipment
Corporation.  These vendors resell and support Banyan's products within their
customer base.  Companies with which Banyan has established support arrangements
include NCR, Cabletron, Dell, Digital, Hewlett Packard, IBM, Proteon, Oracle,
Compaq, Cisco and Microsoft.  Under these alliances, Banyan and its  partners
exchange regularly updated technical information and offer mutual access to
technical support centers, training and advanced technical engineers in order to
facilitate problem resolution for customers with multi-vendor  networks.

Delivery lead times for the Company's products are typically short and,
consequently, substantially all of the Company's revenues in each quarter result
from orders received in that quarter.  Accordingly, the Company does not
maintain any significant backlog and believes that its backlog at any given
point in time is not a reliable indicator of future sales or earnings.  The
absence of significant backlog may contribute to unpredictability in the
Company's results of operations.  The Company's backlog of product orders at
December 31, 1997 and December 31, 1996 was immaterial.

PRODUCT DEVELOPMENT

The Company believes that its future success will depend in large part on its
ability to enhance and broaden its existing product lines to meet the evolving
needs of the enterprise network computing market and to continue developing and
delivering new open standards-based product offerings, including Internet-
related products. The Company's development efforts are currently focused on
enhancing the performance and functionality of its existing StreetTalk for
Windows 

                                      -9-
<PAGE>
 
NT, VINES, Intelligent Messaging and BeyondMail families of products, expanding
its offering of standards based enterprise network services, and new products
and applications for the Internet market, particularly Switchboard. In addition,
the Company anticipates expending significant resources in the continued
translation and localization of its product offerings in 1998.

As of December 31, 1997, Banyan had approximately 98 employees engaged in
research and development, test and quality assurance and technical
documentation. In 1997, 1996 and 1995, Banyan's product development expenditures
totaled $15.9 million, $21.9 million and $24.5 million, respectively.  During
1998, the Company anticipates continued significant investment in product
development.

MANUFACTURING

The Company's software products are distributed as object code on CD-ROM and
standard magnetic diskettes.  Included with the software products are security
codes and documentation, which is available on CD-ROM as well as in print.  In
general, the Company duplicates all software diskettes in house.  Most of the
CD-ROM replication is done by outside vendors, with the ability to produce
smaller volume production runs in-house. From its Manufacturing & Distribution
facility in Westboro, Massachusetts, Banyan ships products worldwide through
authorized network integrators, resellers and international distributors.

COMPETITION

The networking software industry is highly competitive and is characterized by
rapidly changing technology and evolving industry standards.  The Company
competes with a number of companies, including Microsoft, Netscape and Novell,
which have substantially greater development, marketing, sales and financial
resources, distribution infrastructure, customer  support organizations and name
recognition than those of the Company. The services business is highly
fragmented with no dominant set of networking integration or consulting
competitors worldwide that the Company directly competes against.

The Company has invested significant resources to develop products to bring the
Company's directory capabilities to Internet users.  In 1996, the Company,
through a majority-owned subsidiary, introduced Switchboard, a directory service
for Internet users.  In the first quarter of 1998, Switchboard, was rated by
Media Metrix as the Internet's tenth most frequently visited web site.
Switchboard expects to increase promotion and seek additional strategic
partnerships.  The Company success will depend in part on its ability to enter
into strategic alliances with other Internet providers.

The Company believes principal competitive factors affecting the market for its
products include product functionality, performance, quality, reliability and
ease of use; quality of customer training and support; vendor reputation, and
price.  The Company believes that competition in the industry is likely to
intensify as current competitors expand their product lines and new companies
enter the market.  The Company's future success will depend in part on its
ability to respond promptly and effectively to the challenges of technological
change, evolving standards and its competitors' innovations by continually
enhancing its own product and support offerings, as well as its marketing
programs.

PROPRIETARY RIGHTS AND LICENSES

The Company does not currently hold any patents and currently relies upon a
combination of copyright, trademark and trade secret laws and contractual
provisions to establish and maintain its proprietary rights to its products.
The Company believes that because of the rapid pace of technological change in
the networking and computer industries, the legal protections for its 

                                      -10-
<PAGE>
 
products are less significant factors in the Company's success than the
knowledge, ability and experience of the Company's employees, the frequency of
product enhancements, and the timeliness and quality of support services
provided by the Company.

EMPLOYEES

At December 31, 1997, the Company employed 410 persons, including 182 in sales,
marketing and related activities, 98 in research and development, 53 in
customer support and education, 15 in manufacturing, and 62 in finance,
administration, and human resources.  The Company has no collective bargaining
agreement with its employees.  The Company believes that its relations with its
employees are good.

                                      -11-
<PAGE>
 
PART II

ITEM 8.  FINANCIAL STATEMENTS AND SUPPLEMENTARY DATA

Information with respect to this item may be found in the Annual Report to
Shareholders and in the consolidated financial statements and schedules referred
to in Item 14 and filed as part of this 10-K. Such information is incorporated
herein by reference.


                                     -12-
<PAGE>
 
PART IV

ITEM 14. EXHIBITS, FINANCIAL STATEMENT SCHEDULES, AND REPORTS ON FORM 8-K

(a)  The following documents are filed as part of or are included in this Annual
     Report on Form 10-K:

     1.   Financial Statements:
          --  Consolidated Balance Sheets as of December 31, 1997 and 1996.
          --  Consolidated Statements of Operations for the years ended December
              31, 1997, 1996, and 1995.
          --  Consolidated Statements of Shareholders' Equity for the years
              ended December 31, 1997, 1996 and 1995.
          --  Consolidated Statements of Cash Flows for the years ended December
              31, 1997, 1996 and 1995.
          --  Notes to Consolidated Financial Statements.
          --  Report of Independent Accountants for the years ended December 
              31, 1997, 1996 and 1995.
          --  Selected Financial Data for the years ended December 31, 1997,
              1996, 1995, 1994, and 1993.

     2.   Financial Statement Schedules:
          --  Report of Independent Accountants for the years ended December 31,
              1997, 1996 and 1995.
          --  Schedule II -- Valuation and Qualifying Accounts.
          --  Schedules other than the one listed above have been omitted since
              they are either not required, not applicable or the information is
              otherwise included.

     3.   Listing of Exhibits:
          --  The Exhibits filed as part of this Annual Report on Form 10-K are
              listed in the Exhibit Index immediately preceding such Exhibits
              and are incorporated herein by reference.


                                     -13-
<PAGE>
 
(b)  Reports on Form 8-K: On December 8, 1997, the Company filed a Form 8-K
     regarding the Company's continued listing on the Nasdaq National Market
     pursuant to Item 5 of Form 8-K. No other reports on Form 8-K were filed by
     the Company during the last quarter of the year ended December 31, 1997

SCHEDULE II

                          BANYAN SYSTEMS INCORPORATED
                       VALUATION AND QUALIFYING ACCOUNTS
             FOR THE YEARS ENDED DECEMBER 31, 1997, 1996 AND 1995
                            (DOLLARS IN THOUSANDS)


<TABLE>
<CAPTION>
                                BALANCE AT      ADDITIONS                       
                               BEGINNING OF  CHARGED TO COSTS                    BALANCE AT  
          DESCRIPTION             PERIOD       AND EXPENSES      DEDUCTIONS     END OF PERIOD
          -----------             ------      ---------------    ----------     -------------

<S>                            <C>           <C>                <C>            <C> 
Year ended December 31, 1997:

   Reserve for price, sales and  $ 7,168          $ 2,448          $ 5,895          $ 3,721
     doubtful account allowance                                                            
                                                                                           
                                                                                           
Year ended December 31, 1996:                                                              
                                                                                           
   Reserve for price, sales and  $ 5,636          $ 8,712          $ 7,180          $ 7,168
     doubtful account allowance                                                            
                                                                                           
                                                                                           
Year ended December 31, 1995:                                                              
                                                                                           
   Reserve for price, sales and  $ 6,596           $ 4,965         $ 5,925          $ 5,636
     doubtful account allowance

</TABLE> 

                                     -14-
<PAGE>
 
                       REPORT OF INDEPENDENT ACCOUNTANTS



Our report on the consolidated financial statements of Banyan Systems
Incorporated has been incorporated by reference in this Form 10-K from page 30
of the 1997 Annual Report to Shareholders of Banyan Systems Incorporated.  In
connection with our audits of such financial statements, we have also audited
the related financial statement schedule in Item 14(a)2 of this Form 10-K.

In our opinion, the financial statement schedule referred to above, when
considered in relation to the basic financial statements taken as a whole
presents fairly, in all material respects, the information required to be
included therein.



                                               /s/ Coopers & Lybrand L.L.P.
                                               ----------------------------
                                               COOPERS & LYBRAND L.L.P.


Boston, Massachusetts
January 27, 1998
except as to the information presented in
Note Q, for which the date is March 6, 1998

                                     -15-
<PAGE>
 
                                   SIGNATURE
                                        

Pursuant to the requirements of Section 13 or 15(d) of the Securities Exchange
Act of 1934, the Registrant has duly caused this report to be signed on its
behalf by the undersigned, thereunto duly authorized.

Dated: April 8, 1998      BANYAN SYSTEMS INCORPORATED


                           /s/ Richard M. Spaulding
                           ------------------------
                           By: Richard M. Spaulding
                           Vice President and Chief Financial Officer


                                     -16-
<PAGE>
 
                EXHIBIT INDEX

EXHIBIT
NUMBER        TITLE OF DOCUMENT
------        -----------------


3.1(2)        Second Amended and Restated Articles of Organization of the
              Company.

3.2(1)        Amended and Restated By-Laws of the Company.

3.3(10)       Certificate of Vote of Directors Establishing a Class or Series of
              Stock

10.1+*        Second Amended and Restated 1984 Incentive Stock Option Plan.

10.2+*        Second Amended and Restated 1984 Non-Qualified Stock Option Plan.

10.3(6)(8)+   1992 Stock Incentive Plan, as amended.

10.4(1)(8)+   1992 Director Stock Option Plan, as amended.

10.6(7)+      Consulting Services Agreement dated as of November 11, 1996
              between the Company and Burton Technology Partners, Ltd.

10.7(7)(8)+   Employment Agreement dated February 4, 1997 between the Company
              and William P. Ferry, as amended.

10.8(1)       Real Estate Sublease dated June 19, 1991, as amended to date,
              between the Company and Sytron Corporation.

10.9(1)       Lease Agreement dated April 21, 1989, as amended to date, between
              the Company and CB Westboro C Limited Partnership. a Texas Limited
              Partnership.

10.9A(3)      Amendment to Lease Agreement dated April 21, 1993 between the
              Company and CB Westboro C Limited Partnership, a Texas Limited
              Partnership.

10.9B(5)      Amendment to Lease Agreement dated April 21, 1993 between the
              Company and Commonwealth Westboro Limited Partnership, a
              Massachusetts Limited Partnership (as successor in interest to CB
              Westboro C Limited Partnership, a Texas Limited Partnership).

10.10(1)      Lease Agreement dated November 14, 1986, as amended to date,
              between the Company and Aetna Real Estate Associated, L.P. (as
              assignee of Flanders Realty Trust).

10.10A(3)     Amendment to Lease Agreement dated April 21, 1993 between the
              Company and Aetna Real Estate, L.P. (as assignee of Flanders
              Realty Trust).

10.10B*       Fifth Lease Extension and Modification Agreement made as of
              October 15, 1997 between Aetna Real Estate Associates and the
              Company.

10.11(1)      Lease dated July 26, 1989 among the Company, Banyan Systems (UK)
              Limited and Grosvenor Square Properties Developments Limited.

10.13(4)      Joint Venture Agreement dated March 22, 1995 between the Company
              and Marubeni Corporation.

10.13A(5)     Joint Venture Agreement dated June 15, 1995 between and among,
              the Company, Marubeni Corporation and NTT Advanced Technology
              Corporation.

                                      -17-
<PAGE>
 
10.14+*       Senior Executive Termination Benefits Agreement dated September
              15, 1997 between the Company and Richard M. Spaulding.

10.15(9)      Loan and Security Agreement dated as of September 4, 1997 by and
              between Foothill Capital Corporation and the Company.

10.16*         Consent and Amendment to Loan and Security Agreement dated as of
               March 5, 1998 by and between Foothill Capital Corporation and the
               Company.

10.17(9)      Securities Issuance Agreement dated as of September 4 1997 by
              and between Foothill Capital Corporation and the Company.

10.18(9)      Form of Warrant issued by and to be issued by the Company to
              Foothill Capital Corporation.

10.18(10)     Preferred Stock and Warrant Purchase Agreement dated as of
              March 5, 1998 between the Company and HarbourVest Partners
              V--Direct Fund L.P. ("HarbourVest").

10.19(10)     Warrant to purchase shares of Series B Convertible Preferred
              Stock issued by the Company to HarbourVest as of March 5, 1998.

10.20(10)     Warrant to purchase shares of Series C Convertible Preferred
              Stock issued by the Company to HarbourVest as of March 5, 1998.

10.21+*       Separation Agreement and Release and Waiver of Claims dated as of
              May 31, 1997 between the Company and David C. Mahoney.

10.22+*       Employment Letter dated as of May 31, 1997 between the Company and
              Robert D. Burke.

10.23+*       Employment Letter dated as of May 31, 1997 between the Company and
              William E. Warner, Jr.

13**          Selected portions of Annual Report to Shareholders for the year
              ended December 31, 1997 (which is not deemed to be "filed" except
              to the extent that portions thereof are expressly incorporated by
              reference in this Annual Report on Form 10-K).

21*           Subsidiaries of the Company.

23**          Consent of Coopers & Lybrand L.L.P.

27.1*         Financial Data Schedule.

27.2*         Restated Financial Data Schedule for the nine months ended
              September 30, 1997.

27.3*         Restated Financial Data Schedule for the six months ended
              June 30, 1997.

27.4*         Restated Financial Data Schedule for the three months ended
              March 31, 1997.

27.5*         Restated Financial Data Schedule for the fiscal year ended
              December 31, 1996.

27.6*         Restated Financial Data Schedule for the nine months ended
              September 30, 1996.

27.7*         Restated Financial Data Schedule for the six months ended
              June 30, 1996.

27.8*         Restated Financial Data Schedule for the three months ended
              March 31, 1996.

                                      -18-
<PAGE>
 
+    Management contract or compensation plan or arrangement required to be
     filed as an exhibit pursuant to Item 14(c) of Form 10-K.
*    Previously filed.
**   Amended and restated as filed herewith.
(1)  Incorporated herein by reference to the exhibits to the Company's
     Registration statement on Form S-1 (File No. 33-49194).
(2)  Incorporated herein by reference to the exhibits to the Company's
     Registration statement on Form S-8 (File No. 33-54140).
(3)  Incorporated herein by reference to the exhibits to the Company's Quarterly
     Report on Form 10-Q for the fiscal quarter ended June 30, 1993.
(4)  Incorporated herein by reference to the exhibits to the Company's Quarterly
     Report on Form 10-Q for the fiscal quarter ended March 31, 1995.
(5)  Incorporated herein by reference to the exhibits to the Company's Quarterly
     Report on Form 10-Q for the fiscal quarter ended September 30, 1995.
(6)  Incorporated herein by reference to the exhibits to the Company's Annual
     Report on Form 10-K for the fiscal year ended December 31, 1995.
(7)  Incorporated herein by reference to the exhibits to the Company's Annual
     Report on Form 10-K for the fiscal year ended December 31, 1996.
(8)  Incorporated herein by reference to the exhibits to the Company's Quarterly
     Report on Form 10-Q for the fiscal year ended June 30, 1997.
(9)  Incorporated herein by reference to the exhibits to the Company's Quarterly
     Report on Form 10-Q for the fiscal year ended September 30, 1997.
(10) Incorporated herein by reference to the exhibits to the Company's Current
     Report on Form 8-K dated March 5, 1998.


                                      -19-