tombcghtech/api_guidelines huggingface dataset:
Performance analytics enable you to ensure your APIs are meeting SLAs and other contractual obligations. Using analytics to gain insight into API usage should help in making future investment decisions. See 4. Life cycle management for more details.
To gather effective analytics, it is important for all consuming applications to have an API key in order to access the API. This allows driving out of values such as consumer location and associated peak periods, and so on.
3.1.8. State
It is important to understand where and where not to maintain state in the architecture. All effort should be made at keeping the API component completely stateless, meaning that every request to the API layer should be treated as a new request and therefore treated in the same way.
This does not prevent keeping information about the customer or the status of their request, it just means that state information is not held in the API component. This type of state information should be limited to the application or data layers.
3.1.9. API design and business process analysis
When performing the analysis required to design and develop an API, consideration should be given to all components in the reference architecture. This will ensure that the needs of all parties are met.
3.2. Categorisation of APIs
It is good practice to consider your APIs in at least 3 distinct categories to help organise your APIs for speed and agility:
System APIs — APIs that unlock data held in legacy systems or deliver access to functions only backend systems can provide. These APIs encapsulate the complexities of interacting with legacy / backend systems
Process APIs — APIs that create business value by breaking down silos and can be composed to work across system APIs in order to achieve a specific business purpose. These APIs are ones that automate 1 or more steps in a business process
Experience APIs — APIs that innovate and target user experience. These APIs are commonly exposed to external consumers as part of a digital product strategy.
These 3 categories provide a way of grouping APIs based on the level of governance needed to manage changes to the APIs. Considerations include areas such as the frequency of change, effort to change, impact of change and ownership. This is shown in table 4.
Table 4: API categories
Frequency of change
Effort of change
Potential impact of change
Experience APIs
< 6 weeks
Process APIs
3–6 months
Line of business
System APIs
> 6 months
Central IT
3.3. Expose data as a set of resources
To identify resources, it is necessary to analyse the business domain and extract the nouns that are most relevant to business needs or, more accurately, to the needs of the application developers and customers. The API needs to be relevant and useful from the perspective of API-consuming application interactions.
Once all the nouns or resources have been identified, then the interactions with the API can be modelled as Hyper Text Transfer Protocol (HTTP) verbs against these nouns. The HTTP verbs may not map nicely, so it may be necessary to approximate.
For example, it is relatively easy to identify the nouns Customer, Address, Account and Teller as resources in a banking domain. The aim is to offer a clean Uniform Resource Locator (URL) for each thing, and each collection of things.
3.4. API publication and access
The only way an external application developer can find and use your API is if the API is published. The ability to discover an API is fundamental to API take-up and innovation, and is one of the main requirements from commercial businesses. Any published API catalogue needs to be application developer focused, as the primary point of discovery is the application developer. This means that an API must be well documented, and provide accurate and up-to-date guidance to application developer communities to encourage take-up.
Application developers need to be registered before they can develop applications that use the API. The process of registering an application against an API or API product should result in the issuance of access credentials.
Every API provider should offer a means of registration for application developers so that the consuming application base can be known and kept informed of events that impact access to APIs. All-consuming applications should be registered with appropriate credentials, as this enables API providers to understand their consuming application landscape.
Additionally, consuming applications that have been written to use an API will require a level of access relevant to that API or API resource, as follows.
Where the API offers access to view public data, application developer registration should be automatically approved but still result in access credentials relevant to the application.
Where the API offers access to view sensitive data, application developer registration should not be automatically approved and should instigate an agency-defined due diligence workflow and will result in access credentials relevant to the application.
Where the API is for development / testing purposes only, registration should be self-service and the ability for an application developer to ‘try out’ the API should be provided. This is commonly known as ‘providing a sandbox’.
Where a customer is allowed to modify data, registration will include obtaining appropriate credentials for that customer, and an internal step within the API provider to define access controls for that customer commonly referred to as authorisation.
3.5. Discovery
API catalogues will be emergent as more and more public-sector APIs are published for use. Currently, MBIE offers an established catalogue of government APIs: API Explorer — api.business.govt.nz. However, it should by no means be the exclusive location, as private sector catalogues such as Programmable Web and Mashape offer good API catalogue offerings.
When using a catalogue to identify an appropriate API, application developers are focused primarily on the resource they wish to access, rather than the government agency providing the resource. This is particularly relevant as governments and agencies change over time — the API product or resource should remain relatively stable.
The business world tends to see government as one large entity holding valuable information, and are likely to know in detail what resource they are looking for but not necessarily which agency that resource would reside within. Hence, it is important to expose APIs in terms of the resources they offer rather than the internal business process or agency structure.
The long-term aim is for discovery of APIs to be automatic and dynamic, enabling real-time integration of resources and onward presentation to customers.
4. Life cycle management
Developing and delivering APIs changes the model for most government organisations in that they are now creating and delivering products in a very similar way to a commercial entity. With that comes a need to manage, maintain and support those products throughout their life cycle in a professional way, otherwise take-up of the APIs will be limited and painful for all concerned.
The standard API life cycle follows these steps:
Before an API goes live, in other words, is released for use, provisions should be in place to:
support the internal development and testing of APIs
handle release management
support onboarding of application developers
define the level of service the application developer (and customer) can expect from the API (SLA)
support application developer usage of the API, including:
encompass life cycle / change management, handling API versioning and retirement management
cater for incident / events
perform API management.
To some extent this mirrors standard service delivery practices defined in the Information Technology Infrastructure Library (ITIL),(1) but with an API-specific flavour.
Figure 7: API service life cycle management
Figure 7 is a circular flow diagram that illustrates the service life cycle management of an API. An outer circle surrounds the entire diagram and represents continual service improvement. At the centre is a circle representing service strategy. Between these 2 layers are the following 3 areas that flow from 1 to the other:
Service design — consumer management, service level management and API design
Service transition — internal development and testing, API catalogue, API release management and life cycle / change management
Service operation — access management, consumer support, incident / event management and API management.
This life cycle is described in more detail in sections 4.1. to 4.4.
View larger image (PNG 109KB)
4.1. Service design
Service design covers the initial analysis and business drivers for the API as well as the way in which it will be built, where it will be hosted, and so on. Service design not only applies to the initial design of the API, but also to any changes and improvements over time.
4.1.1. Consumer management
When designing an API, it is important to inform all potential application developers as to the thinking behind the API, including the resources to be exposed, the granularity of access, and capabilities to be offered. This can include roadmaps to indicate the planned evolution of an API so that developers are pre-warned and have time to prepare for changes.
Ideally, the API services should be designed in collaboration with application developers to ensure the right product is developed to support diverse business processes across a broad set of consuming applications. It is best if API development can be performed in coordination with application developers who are producing consuming applications.
Identifying potential application developers and understanding the demand from the market can be difficult. Careful consideration should be given as to the best approach to engaging with the market and identifying your potential application developer community.
4.1.2. Service-level management
Without robust service-level management, it will be hard to engender trust in government APIs, which will negatively impact uptake.
Application developers (especially commercial entities) will need to know how long the API will exist, what commitment there is to its availability and performance, and what support is offered to those who consume the API.
Without this, API usage will be based on an untrusted model, where application developers prepare for the API being unavailable. This results in consuming applications using APIs to top up local caches of data, or to support existing batch processes, missing out on the real-time benefits of APIs.
Support could involve multiple platforms (some cloud-based, perhaps) and multiple vendors, and therefore the service levels offered in the SLA may be limited to that of the lowest common denominator.
4.1.3. API design
This incorporates the following tasks that are involved in the design of the API, prior to implementation:
business analysis of the target audience, the business processes potential application developers are trying to implement, and how the API fits into those business processes
Agile processes for data analysis, resource identification, software design for development of the API
infrastructure planning, for the hosting of the API.

Note: It is worth recognising that for application developer support, an accessible test platform is just as important as the production platform that will host the final API
responding to requests for additions and modifications from application developers via consumer support — so service analysis does not just apply to initial design, but also to any changes.
4.2. Service transition
Service transition focuses on the implementation build of the API and deploying the API for operational use.
4.2.1. Internal development and testing
APIs need to be developed in a collaborative, flexible and adaptive way. Once the API interface specification is agreed, it is more important to get something developed, however small, and get that published for application developers than to try to build a complete API product for release.
It may well be that initial versions of the API have functionality stubbed out (resources that can be called but return canned responses) or missing, which is valid so long as the documentation indicates this. Support for this form of iterative development can be enabled through the following:
Agile practices — Agile practices ideally suit this form of iterative development as they focus on developing small, incremental releases, ‘failing fast’ (finding out what is wrong early rather than too late) and frequent delivery of products
Configuration management — all the components that make up an instance of the API should be held within version control, so that it is possible to rebuild a previous version, if necessary. This involves:
version controlling API interface specifications
version controlling the associated API code
keeping track of dependencies (for example, external libraries being used within the API code)
making sure access policies for individual consumers are version controlled
being able to provably recover all the elements of previous iterations of APIs and rebuild or redeploy if required
Automated testing — to make the incremental release pattern efficient, it is advisable to develop automated tests in conjunction with the API code so that testing becomes an intrinsic part of the build process
Continuous integration — tools exist that make it possible to spawn automated build, testing and subsequent deployment whenever an update of the code is placed into configuration management.
4.2.2. API catalogue
Before an API is released for application developers to access, the API’s description and interface specification should be published to an API catalogue.
The API catalogue will contain a list of all the APIs offered, along with their interface specifications and guidance on how to gain access, and use the APIs, including the granularity of access control. This information needs to be up to date and accurate in order to be relied upon by API application developers.
4.2.3. API release management
Release management is an important aspect of API transition. The aim with API development is to make small changes and release often.
The release management aspects include:
versioning — informing application developers about the following changes:
where the interface specification has been changed, a major version release is required with appropriate warning to application developers, scheduled deprecation of the previous major version and support for migration to the new version
where backend changes that have little or no impact on the interface specification, and should have minimal impact on consumers, a minor version change release is appropriate
when the API changes, the interface specification must also be changed to reflect the changes. Hence, it is important to know who your application developers are
planning API rollout — ensuring all the API artefacts are rolled out effectively and on time to the correct platforms
emergency patches— informing application developers as to the need and schedule for emergency patch rollout, and ensuring emergency patching does not impede consuming applications.
Releases should be made to a test or development environment first.
4.2.4. Life cycle / change management
An API has its own life cycle, and it needs to be managed through the whole of its life cycle, from creation to destruction. This involves:
monitoring performance of the API to make sure it adheres to the SLA offered (for example, availability, signs of increasing demand requiring horizontal scaling)
monitoring usage to make sure an API, or API version, is only retired when the majority of consuming applications have migrated off it
trimming APIs — removing capabilities that are unused, have never been used and are not likely to be used
ensuring the API roadmap is up to date and gives a good indication of when major changes are scheduled to be made to the API.
4.3. Service operation
The service operation part of the cycle looks at the actual delivery of the services to the service levels advertised. It handles management of, and access to, the API and any underlying applications, and looks after the infrastructure that underpins it. It also includes consumer support and incident management.
4.3.1. Access management
The first interaction that service operations is likely to have with APIs is the initial act of onboarding application developers. Onboarding should provide everything the application developer needs in order to interact with the API, including access to a test environment running a representative copy of the API, underpinned by test data, documentation and preferably examples to work with.
Consideration may be needed as to whether all application developers can work with shared test data or if they need individual test data specific to their purposes.
In their day-to-day development activities application developers will want to be able to test the API without making (computationally or potentially financially) costly call outs to third party services, so it may be beneficial to provide mock versions of those APIs specifically for testing purposes.
For full system tests, however, application developers will want their applications to test the full flow including any third-party service, so an automated mechanism for that may need to be built.
Access management includes providing mechanisms through which application developers can apply for, and receive, permission and associated details to use the API. This includes external developers who want to build applications that make use of the API and applications that will ultimately be integrated with the API.
It also covers managing access to the API, including specific, fine-grained control for consuming applications. This allows operations to deploy access policies to ensure a consuming application’s access to the API is in line with agreed constraints.
4.3.2. Consumer support
Application developers will need a variety of support mechanisms to aid their use of an API, including:
getting set up to use the API
understanding what to use during development
support for testing of their application in use of the API
reporting issues with the API.
These mechanisms should include:
telephone support
support desk email address
online forum / support community (see also 5.1. API components).
It could also include interactive real-time support
It is useful to include indications of the level of support so that application developers know which form of support will most rapidly address their issue — for example, 9am to 5pm telephone support, 24-hours-a-day, 7-days-a-week community forum, and response times to API failure reports.
There should also be support for handling requests for change, modifications or additions needed to the API. It should also be possible to capture and handle requests for new APIs.
4.3.3. Incident / event management
Incident and event management is geared around events picked up through monitoring, and unplanned incidents, and involves substantial amounts of communication. This includes:
monitoring the system as a whole to identify potential issues and pre-emptively apply mitigation — for example, throttling to counter potential Denial of Service (DoS) attacks
capturing reports of, then informing application developers of, unforeseen incidents that are currently causing a disruption of service
informing application developers of remediation of incidents, including resolution plans and predicted completion times
deploying temporary fixes, if necessary.
4.3.4. API management
APIs need to be managed as products, very similar to software products that commercial entities release. API management needs to be consistent across all the APIs the provider is publishing.
API management looks after availability of the API. This can involve throttling to make sure all consumers can get access to the API within the bounds of the SLA. It can also include quota management, whereby consuming applications are given limited access (for example, a set number of calls per hour) to protect the API from abuse or overuse. It should be possible to use analytics (see 4.4.1. Analytics) to assess whether throttling or quotas are needed.
For more information about capabilities to support this activity, see 5.1. API components.
4.4. Continual service improvement
It is not sufficient to release an API and let service operation keep it running. There is also a need to ensure that APIs are running optimally and can adapt to changing demand over time.
4.4.1. Analytics
Capturing and analysing data about an API in operation will pull out information useful to adapting to changes in demand. It is therefore useful to gather analytical data around:
take-up metrics, end-user analytics such as location
tracking API consumers, their registrations and API usage
API performance — identifying most commonly used APIs calls so they can be made efficient
event behaviour (for example, common patterns of behaviour)
trace and diagnostics data.
From an API take-up / consumption perspective it is useful to capture who, where, when, how, how often, and what device type is being used. Analysis of this data can then be used to demonstrate Return on Investment (ROI).
Performance metrics are also useful, such as error rate, throughput, response time, transaction speed, backend performance, cache performance. These values can help identify trends and bottlenecks.
4.4.2. API availability
It is important not just to monitor and gather operational data about running APIs, but also to use that information to improve API offerings.
API availability is of utmost importance to API consumers, so the API providers need to monitor usage and respond dynamically to increases in demand. This requires transaction-throughput monitoring to identify potential bottlenecks or overloaded APIs and to ramp up availability to meet demand.
5. Getting started with APIs
When getting started with API development and delivery the following approach is recommended:
Start small — pick one aspect or API product from the agency or information source and build a well-defined API.
Work with a single application developer to ensure the interface specification meets their needs while still having relevance to broader use (for example, is not aligned solely to their business objectives).
Have the following basic capabilities set up to support development, discovery of and access to your API:
an Application Developer Support Environment — developer portal tools can aid in this
API gateway — for ensuring safe access to your APIs
some means of managing published APIs — an API manager can assist in this.
For development and hosting of the API, consider using off-the-shelf (OTS) products or software as a service (SaaS), which is subscription-based (for example, based on the number of transactions) to reduce maintenance and support costs. Subscription-based offerings allow an agency to start with a small number of transactions and increase these over time as their API ecosystem grows.
Use analytics to gain visibility of take-up, developer engagement and usage profiles, and adapt APIs to meet changing needs over time.
5.1. API components
The following are definitions of components that provide API development support capabilities.
Figure 8: API core components and capabilities
Figure 8 illustrates a general API architecture that includes the core components and capabilities of an API. In general, the diagram shows:
consuming application developers discover and subscribe to APIs through the API developer portal. They then create applications (for example, web, mobile and server apps) that sit on devices.
customers use consuming applications on devices (for example, desktop browsers, mobiles and web servers) to access an API from the API gateway.
Identity Federation Services provides authentication and Single Sign-On (SSO) services to customers and secure transportation of authentication and authorisation information.
the API gateway hosts the API after it is published by the API manager
internal API developers and application developers expose and create APIs through the API developer portal, API manager and API gateway.
The main components and capabilities are described in more detail in sections 5.1.1. to 5.1.3.