# Agile business intelligence

Agile business intelligence (ABI) refers to the use of agile software development for business intelligence projects. Factors considered important for the success of ABI projects include a holistic approach to BI architectures, organizational forms, technologies, and the use of agile process models adapted to BI.
Agile methodology operates on an iterative principle, providing new features to end users sooner than the traditional waterfall processes, which delivers only the final product at the end of the project. With this technique, the requirements and design phases overlap with development, thus reducing the development cycles to achieve quicker delivery. It allows for adaptive planning, evolutionary development and delivery, a time-boxed iterative approach, and responsiveness to change. A key focus of ABI is delivering BI functionality in increments via shorter development cycles and documenting each cycle as it happens.
Agile business intelligence is a process that allows managers to access product data for informed decision-making through development using agile methodology. ABI employs Agile techniques in the development of BI applications, such as dashboards, balanced scorecards, reports, and analytic applications.
According to research by the Aberdeen Group, organizations with established ABI implementations are more likely to have processes in place for ensuring that business needs are being met. However, they note that end-user participation and "frequent collaboration between IT and the business" are critical to ABI implementation success.

## Definition

Agile business intelligence (ABI) is a methodology that applies agile software development principles to business intelligence (BI). In particular, ABI encourages flexibility, iterative improvement and adaptability in response to changing environments.

## Key performance criteria

Aberdeen's Maturity Class Framework uses three key performance criteria:
* Availability of timely management information: IT should provide timely and accurate information to business managers to enable sound business decisions.
* Average time required to add a column to an existing report: The time required to modify an existing report by adding a column is measured because if information cannot be obtained within the time required to support a decision, it has no value.
* Average time required to create a new dashboard: The time required to access any new or updated information is measured by calculating the time required to create a new dashboard.

## Elements of ABI

Margherita Bruni described five elements of an ABI enterprise environment:
* Agile Development Methodology: An agile, iterative process shortens development cycles, speeding up the time to market for BI requests.
* Agile Project Management Methodology: Continuous planning and execution, where planning is done at the beginning of each cycle, allows the scope to be changed during the development phase.
* Agile Infrastructure: The system should have virtualization and horizontal scaling capability, providing the flexibility to modify the infrastructure and maintain near-real-time BI more easily than the standard Extract, transform, load (ETL) model.
* Cloud & Agile BI: Bruni suggests that companies in their initial stages of implementing ABI should consider using cloud technology, as it may be cheaper to store and transfer data.
* IT Organization & Agile BI: To achieve agility, the IT team should interact with the business, address business problems, and maintain a strong and cohesive team.

## BI, BI Model, and its characteristic goals

Wayne Kernochan of InfoStructure Associates conducted a two-year study of BI processes across several businesses and developed the following model and its goals:
* Data entry — accuracy
* Data consolidation — consistency
* Data aggregation — scope
* Information targeting — fit
* Information delivery — timeliness
* Information analysis — analytic ability
Kernochan also identified these common issues with current BI processes:
* 20% of data contains errors (accuracy)
* 50% of data is inconsistent (consistency)
* It typically takes 7 days to get data to the end user (timeliness)
* It isn't possible to do a cross-database query on 70% of company data (scope)
* Executives don't receive the data they need 65% of the time (fit)
* 60% of the time, users can't do immediate online analysis of data they receive (analyse ability)
* 75% of new key information sources that surface on the Web are not passed on to users within the year (agility)
The study concluded that applying ABI in existing business intelligence minimizes these problems, leading organizations to slowly transition their processes to agile methodology and development.

## Improving business intelligence agility
### Data entry

Data is often inaccurate (20% error rate) and inconsistent (50% rate). These numbers increase with new types of data, necessitating re-evaluation and correction of processes to minimize data entry errors.

### Data consolidation

Companies often have multiple data stores, and data is scattered across them. Agile methodology motivates the auto-discovery of new data sources and automated upgrade of metadata repositories to accommodate new information.

### Data aggregation

Data aggregation is the process in which information from many data stores is pulled and displayed in a summary report. Online analytical processing (OLAP) tools are commonly used for this.

### Information delivery

A key principle of ABI is to deliver the right data at the right time to the right individual, while maintaining historical data for comparing current performance with past performance.

### Information analysis

Another key principle of ABI is improving the decision-making of its users by focusing on analysis tools that improve an operational process or new product development.

## ABI checklist

* Assemble a team of developers and business representatives who will work together.
* Select either a business stakeholder or technical liaisons to represent the business.
* Identify and prioritize user stories or requirements to address during an initial project.
* Assess ABI delivery tools that can integrate with your existing data warehouse and BI environment.
* Initiate the iterative development process.

## Advantages of using ABI

ABI is intended to drive its users to self-serve BI, offering organizations flexibility in terms of delivery, user adoption, and return on investment.

### Faster to Deliver

Using Agile methodology, the product is delivered in shorter development cycles with multiple iterations. Each iteration results in working software that can be deployed to production.

### Increased user acceptance

In an Agile development environment, IT and business personnel work together, often in the same room, refining business needs in each iteration. This can increase end-user engagement by emphasizing the changing needs of non-technical business users.

### Increased return on investment

Organizations can achieve an increased rate-of-return with shorter development cycles by minimizing IT resources and time while delivering relevant reports to end-users.

## ABI best practices

* Create a program charter to set stakeholder expectations on how the ABI system will work.
* Start with the business information that needs to provide context for scope.
* Time iterations.
* Emphasize data discovery through the requirements and design phases.
* Use the Agile process of incremental and iterative development and deployment.
* Validate the BI architecture and get approval on the proof of concept.
* Complete data validation and verification for each development iteration.
* Use flow charts or diagrams to explain the BI process, along with some documentation.
* Test any change that will be deployed to production in a regression environment.
* Have a formal change control to minimize risk, as all changes must be approved before going into production.