tombcghtech/api_guidelines huggingface dataset:
API guidelines — Part A: API concepts and management 2022
APIs (Application Programming Interfaces) enable reuse of data, transactions and rules. This creates an opportunity for greater cooperation and interoperability between agencies to dramatically improve service design and delivery.
1. API concepts
1.1. Why an API
Interoperability between government agency systems and customers today involves a wide complex range of mechanisms, protocols, standards, technology and so on, and the idea of using APIs is to decouple the interface from this complexity as much as possible.
A clearly defined API specification provides a common language and ensures a common understanding of the interaction. So, with APIs, the consuming application only needs to understand the interface and does not need to understand the underlying technologies. This greatly simplifies the interaction and encourages interoperability.
1.2. What is an API
An API is a computing interface that defines a shared boundary across which 2 or more components exchange information. In practice, an API is generally a piece of software that provides a way for other disparate pieces of software (applications, systems) to talk to one another. It can be considered as a:
means of requesting and retrieving information from a system or server so that it can then be presented to the user in a context / process-specific way — for example, in a mobile application or web application
way of de-coupling system-to-system interactions
contract between the application developer and the provider describing what information or functionality they can gain access to and how that access is provided.
APIs are commonly categorised as shown in table 1:
Table 1: Common categories of API
API category
An API that is used solely within your organisation by known internal personnel (employees or contractors). For example, an API that provides revenue forecasting information.
An API that is used by customers and / or application developers who are outside your organisation.
Partner API
An API that is used by business partners who are known to your organisation and with whom you often have a special contractual relationship. For example, an API that provides sales information to the partner.
Public or open API
An API that is used by anyone who wants to use it, as long as they can meet your access control requirements.
1.3. Basic API architecture
Figure 1 shows the basic architecture of an API and its use.
Figure 1: Basic API architecture and use
Detailed description of figure
Figure 1 shows the basic architecture of an API. The diagram illustrates the following workflow processes:
An API provider (for example, a government agency) owns the resource (information or functionality) in applications that will be exposed by the API.
API developers build the API — they start by defining the interface specification, and then they build the API.
Application developers create new software, or adapt existing software, for devices such as smartphones, computers and appliances, to use the API. This software, known as an API consuming application, uses the API to gain access to the resources the API provider is offering.
Finally, customers use the API consuming applications. The application developers, the API consuming applications and the customers are referred to as API consumers.
View larger image (PNG 114KB)
Example: Funded healthcare eligibility check
The Ministry of Health (MoH) recognises there is demand for a system to help general practitioners (GPs) determine if a patient is eligible for funded healthcare or not. This will help reduce administration costs for GPs and make it easier for patients to understand their financial obligations for health services. The process for determining eligibility involves:
checking the MoH National Health Index to see if the patient has been previously enrolled with another GP and, if not,
checking with the Department of Internal Affairs to see if the patient is a NZ citizen and, if not,
checking with the Ministry of Business, Innovation and Employment (MBIE) to confirm the patient’s immigration status.
Figure 2 illustrates the API architecture for the example of a funded healthcare eligibility check. Table 2 details the API roles for this example.
Figure 2: Example of API architecture for a funded healthcare eligibility check
Figure 2 shows the API architecture for the example of a funded healthcare eligibility check. The diagram illustrates the following workflow processes:
The Ministry of Health (MoH) is the API provider. MoH has developed an automated eligibility check application that determines whether a patient is eligible for funded healthcare or not.
The MoH API developer develops an API that takes the information from the automated eligibility check application to a GP’s Practice Management System (PMS).
The PMS developers (application developers) enhance their PMS system to call the API and develop business rules for what to do if a patient is not eligible for funded healthcare.
The GP (customer) uses the PMS (consuming application) which uses the API to determine whether the patient is eligible for funded healthcare and to decide how much to charge the patient for their visit.
View larger image (PNG 131KB)
Table 2: API roles in funded healthcare eligibility check example
API role or component
API provider
MoH develops an eligibility check application that collaborates with government agencies, such as the Department of Internal Affairs (DIA) — to determine citizenship, and MBIE — to check immigration status, in order to determine whether a patient is eligible for funded healthcare or not.
The automated eligibility check process.
API developer
MoH develops an API to allow the system used by GPs, called the Practice Management System (PMS), to determine if the patient presenting is eligible for funded healthcare.
Application developer
PMS developers enhance their systems to call the API and develop business rules for what to do if a patient is not eligible for funded healthcare.
Consuming application
PMS at the GP office use the API to determine whether the patient is eligible for funded healthcare and to decide how much to charge the patient for their visit.
GPs use their PMS system.
The Practice Management System, the PMS developers, and the GPs using the PMS.
1.4. API privacy
Clearly, some APIs will be for internal government use while others will be for external or public use. The types of information offered by these different types of API will be limited by the privacy act and security concerns.
Privacy laws that define the management of customers’ information and the related consent policies are now at the forefront of many organisations’ obligations as legislation mandates how this is applied along with the related penalties.
Globally there are a number of privacy laws that stipulate strict compliance for organisations within their jurisdictions, but it is important to note that these laws can apply to organisations outside the physical borders, for example the Europe Union’s (EU) General Data Protection Regulation (GDPR) covers any global organisation that might deal with a customer who resides in the EU.
New Zealand privacy legislation is dictated by the Privacy Act 2020.
Privacy Act 2020 — New Zealand Legislation
The following global privacy laws should be considered if an organisation manages customers who reside outside New Zealand.
Table 3: Global privacy laws
Privacy Law
General Data Protection Regulation (GDPR)
Complete guide to GDPR compliance — GDPR Compliance Guidelines
Children’s Online Privacy Protection Act (COPPA)
Children’s Online Privacy Protection Rule — Federal Trade Commission
California Consumer Privacy Act (CCPA) — State of California Department of Justice
1.5. Needs and drivers
So why build an API? In what situation is an API the right thing to develop? There is an increasing drive towards government agencies delivering APIs as part of open data (Data.govt.nz) and ease of digital interaction with government. But different government organisations have the following different drivers.
Moving towards a collaborative model of service delivery — enabling agencies to participate in external business processes.
Improving customer interactions with government — enabling integration with partner agencies at a system level rather than via humans.
Moving from retailer to wholesaler of services — supporting integration with third parties and enabling them (rather than the agency) to build the customer-facing aspects.
Using cloud services — ensuring reliable and secure information exchange with cloud-based, commercial off-the-shelf (COTS) solutions.
Improving access to information — providing a more robust interaction model than the current practice of data and screen scraping.
Meeting the goals of open data — using APIs to open public data to public access, enabling the private sector to create their own solutions based on public data.
Improving quality of information — providing a reliable single source of the truth and a more robust interaction model than the current practice of regular bulk information sharing.
Exploiting emerging technology — encouraging uptake of new and innovative technologies to keep pace with customer demands and expectations.
Enabling channel shift — supporting agencies in their desire to make APIs their main web channel in preference to developing agency‐specific web user interfaces (UIs).
Contributing to the business growth agenda — helping to grow competitive businesses within New Zealand in the information-based economy.
1.6. Why have API guidelines
While some NZ public sector organisations have been producing APIs for a while, others are taking their first steps in offering APIs for public use. It is useful to provide a reference and set of guidelines to support this API development in order to:
encourage consistency across government API offerings and thus making it easier for third parties to build solutions using government APIs
avoid different government organisations developing or delivering idiosyncratic APIs
generate awareness of what is needed to support APIs as a delivered product to private sector customers — for example, levels of service and support capabilities
capture API best practice for reference by the API development community, indicating where commonality is required, and which areas allow for more flexibility
reduce disruption to application developers as government APIs evolve
reduce effort for agencies by simplifying the process of developing and delivering on their API strategies.
The Strategy for a Digital Public Service recognises the need to collaborate and co-create with partners from inside and outside government, including the private sector and Māori, in order to unlock the value of information to help solve complex problems and generate innovative ideas.
The development of APIs is considered a key part of this picture in order to improve access to public services and deliver a joined-up experience to the customer. This approach requires collaborative service delivery, where front-end and backend capabilities may be developed separately — partly by the government sector and partly by commercial entities or NGOs — and delivered in a coordinated fashion to offer a seamless experience to the customer. This change in approach can obviously impact the business function of an agency.
An NZ government example of collaborative service delivery is the Inland Revenue (IR) service that allows customers to file their GST returns directly through some software packages. IR worked closely with software package providers to design this service, and software package providers extended their packages to use this service.
For customers using this capability, their GST return information is prepopulated from the sale and purchase figures from the software package. It all happens in a seamless interaction from the customer’s perspective, reducing the amount of time they spend double handling their accounts information and reducing opportunities for error.
If we look at API delivery in an all-of-government context, we see that the initial focus tends to be within an agency’s sphere of influence, as part of the business systems they deliver and processes they perform. APIs are seen as a means to an end, quite often driven from a technological perspective.
While organisations across government have different levels of take-up in terms of APIs, there is a common pattern of evolution, which can help when developing an API strategy and road mapping delivery of APIs.
1.7. API evolution
An organisation’s development of APIs tends to start from an internally-focused perspective that concentrates on existing internal resources (data, functionality). Over time this perspective broadens to encompass the needs of their sector, then all-of-government and beyond. A potential roadmap for the evolution of government APIs over time is illustrated in 5 main stages in Figure 3, and aligned to examples from mobile banking’s evolution.
Figure 3: API evolution
Figure 3 is a flow diagram that illustrates (from top to bottom) the evolution of government APIs, from internally-focused APIs to future capability, and compared with examples from the evolution of mobile banking. The main stages are as follows:
Parochial:
Read information
Request action
Example: mobile banking — visibility of bank balance.
Integrate:
Inter-agency process support
Example: mobile banking — real-time bank balance plus pay existing payees.
Create new info
Update existing info
Example: mobile banking — create and pay new payees, and update contact details.
Primary channel
APIs are the primary source
Reduce user interface (UI) development
Example: mobile banking — all banking capabilities available (via API).
Automatic:
Machine discoverable
Machine consumable
Example: mobile banking — unknown.
The stages are described in more detail in sections 1.7.1. to 1.7.5.
View larger image (PNG 60KB)
1.7.1. Parochial
Initially, parochial APIs are developed to enable programmatic access to existing information stores (for example, ‘see radio spectrum usage’) or to request an action from an agency, which spawns an internal business process (for example, ‘request patent renewal’). These are built from the perspective of the resources the agency holds, not from an all-of-government perspective.
High transaction volumes are often a driver for agencies developing parochial APIs. The APIs are designed either for consumption by systems within the organisation or by selected consuming applications outside the agency. Access to information may also support some filtering (for example, ‘view available or free radio frequencies’). This step of the evolution may be bypassed by organisations whose drivers are focused on external integration.
1.7.2. Integration
Opening up APIs for integration is the next step, where business processes that span government organisations are acknowledged and supported. This often encompasses a move from batch processing to real time data update, with the associated mindset and business process change.
APIs form another digital channel, alongside the standard web user UI channel that supports the majority of interactions, with the paper channel being retained only to support exceptions.
1.7.3. Modify
APIs are then created that start to support authorised modification or addition of information to information stores. This is either to support direct update by the end customer (via web form), or by third parties acting on their behalf (with delegated authority — for example, mobile apps). APIs are at this point becoming one of the core products that an agency releases, offering transaction handling and taking over from web interfaces.
Ideally private sector organisations increasingly use the APIs to incorporate public sector capabilities into their commercial offerings, migrating government from the role of retailer to that of wholesaler.
At this stage of the API evolution, it is important to maintain and govern a resource model to ensure consistency across APIs, and potentially across agencies, if a sufficient number of agencies have reached this level.
1.7.4. Primary channel
Next, APIs become the primary channel for information and request for action. This fosters innovation by opening up resources to third parties who have not previously had access, such as commercial entities and NGOs.
This stage of API evolution starts unlocking the value of information and enabling compliance or interaction with government processes to be built invisibly into everyday applications (for example, automatic GST returns via accounting software). It also moves the onus of UI development from government to the marketplace, allowing new and intuitive means of access from mobile devices, smart phones, domestic appliances and so on, to be created.
At this stage, it is likely the marketplace will encourage government to start developing a shared resource model to help make it easier to consume APIs from different agencies.
1.7.5. Automatic
Finally, we move to the automatic use of APIs, where APIs are both machine discoverable and consumable, requiring minimal developer input for effective delivery of capability to the customer. It is difficult to predict the impact this will have, as there are no examples to date of this kind of capability.
1.8. Business change
The building of an API channel opens up an agency to business process change that influences the ways they work internally and with partner organisations. It can also alter the main point of contact with the public, or add a layer of abstraction, as API-consuming applications replace direct contact with customers. Hence, change management will be paramount in the development of this new API channel.
The Ministry for Primary Industries (MPI) is looking to develop an API to assist animal status declaration, which keeps track of cattle and sheep transportation. Currently, animal transportation information is captured in paper form by the farmer or transporter, which must be produced on demand when MPI needs the information for a biosecurity response.
However, if the information can be captured via mobile application and passed to MPI through integration with the farm management system, then the information is more trustworthy and will be readily available to support associated MPI business processes.
Physical spot checks could become a thing of the past — MPI will no longer have direct contact with the customer (farmer or transporter), only with the API consuming applications (farm management systems).
This has an impact on internal team make up, working practices, travel and interactions. It is important when designing APIs to consider the impact on the business and to put in place the change support that will inevitably be needed to underpin controlled business process change.
While most government organisations are looking at API development from the perspective of their own needs or the needs of their sector, there is an emerging consideration of APIs in the context of all-of-government integration (federated service delivery) and joined-up capability to support customer-centric public services.
Once the fundamental building block APIs are in place to assist with integration and information sharing, it enables commercial operators to innovate and combine multiple services to provide value-add customer-focused end products. Is it possible to foresee a future where public interactions with disparate government agencies are automatic, seamless, and built around the end user’s ultimate goal (for example, setting up a business)?
Currently, as shown in figure 4, a member of the public acts as the integration point between agencies, supplying the same or similar information to each agency (name, contact details) while trying to achieve their objective and having to develop a mental model of each agency’s business practices to navigate by.
Figure 4: Integration by humans
Figure 4 is derived from Federated Service Delivery (FSD) reference architecture. It shows a member of the public (a customer) in the middle of the diagram acting as the main integration point between agencies. The customer is supplying the same or similar information to 4 agencies through the agency interfaces, which could be by human interaction, paper or web-based forms or mobile applications, to achieve their objective. Meanwhile, a thought bubble coming out of the customer shows they also have to develop a mental model of each agency’s business practices to navigate the process.
View larger image (PNG 71KB)
Figure 5 shows that with the gradual emergence of an API layer across government, the need for this human information sharing mechanism should reduce, moving the integration into the system layer. Coordination across this layer will be fundamental to seamless delivery of federated services.
Figure 5: Integration by APIs
Figure 5 is derived from FSD reference architecture. Compared to in figure 4, the customer is no longer the central point of integration between agencies. There is 1 connection point from the customer to the application layer, which includes various devices such as smartphones, tablets and desktop computers. These consuming applications perform the integration between the 4 agencies on behalf of the customer. The agencies in turn offer APIs to the application developers through the API layer.
View larger image (PNG 48KB)
There is also a move towards government as the wholesaler rather than the retailer — in other words, provider of the information at the programmatic layer (API) rather than at the presentation layer (for example, website). If the APIs were available, software packages that customers already use could be enhanced to complete many of the obligatory government interactions automatically, in the background (with customer permission).
The commercial world is seen as having the means, experience and drivers to develop effective ways of interacting with the public — so, by opening up resources via APIs, government agencies can encourage innovation and create opportunities for more intuitive and supportive channels of working for the customer.
MBIE currently provides APIs that allow third parties to manage their customer’s NZ Business Number (NZBN) primary business data and is working on a business registration service that allows business customers to complete their common government registrations in one place.
There will still be instances where government needs to act as an impartial intermediary and offer capabilities that are unbiased and independent (for example, patent examination) or offer basic capabilities for organisations that cannot afford anything complex, but these should be the exception rather than the norm.
It is also recommended that any action that legally commits the customer (for example, GST return submission) requires a manual step so that such commitments are customer controlled.
1.9. Potential impacts
1.9.1. Potential policy impacts
Legislation and regulations — APIs must align to the legislative or regulatory requirements on the agency. In some cases, legislation and / or regulations may limit the opportunities for agencies to use APIs. For example, legislation may not allow for electronic submission. Conversely, an API may introduce new risks that are best managed through legislation or regulation. For example, if there is a bug in a software package, which means the information submitted electronically is incorrect, who is liable for the error and the resulting penalties or criminal offence, if any?
Privacy — Another area of legislation that APIs must align with is the Privacy Act 2020. Agencies with APIs that involve personal information will need to consider the privacy implications. If appropriate, an agency may choose to undertake a privacy impact assessment to understand the privacy implications of an API. For example, the API may introduce the possibility for amalgamating information in ways unpredicted by the agency. Additionally, the Privacy Act may impose limitations on APIs. For example, the methods available for uniquely identifying individual customers are limited by the Privacy Act.