# Presentation: 189611

## Modeling Banks’ Payment Submittal Decisions

- Walt Beyeler1
- Kimmo Soramäki2
- Morten L. Bech3
- Robert J. Glass1
- 1Sandia National Laboratories
- 2European Central Bank
- 3Federal Reserve Bank of New York

- PAYMENT AND SETTLEMENT SIMULATION SEMINAR AND WORKSHOP
- Helsinki                August 2005

- The views expressed in this presentation do not necessarily reflect those of the Federal Reserve Bank of New York or the Federal Reserve System

## Orientation

- NISAC is a core partnership of Sandia National Laboratories (SNL) and Los Alamos National Laboratory (LANL), and is sponsored by the Department of Homeland Security's (DHS) Information Analysis and Infrastructure Protection Directorate.
- NISAC program is charged with understanding 14 critical infrastructures and their interactions for U.S. DHS
- We depend on engaging experts who design and operate infrastructures.  We've been especially fortunate in developing contacts in banking and finance
- We look for models that capture common features of many infrastructures, and are therefore more abstract than industry models

## Outline

- Goal
- Model Design
- Formulations of Payment and Funding Decision Rules
- Results
- Future Work

## Project Goals

- Understand possible responses to unusual conditions
- Try to capture the complex dynamics as adaptive responses to constraints
- Does the ability to adapt make systems more robust?
- Are adapted states especially dependent on specific constraints or regularities?
- Is adaptation itself a source of novel conditions?

## Polynet Model Features

- Designed to support models of diverse systems characterized by network interactions
- Defines supporting classes which can be extended and specialized
- Draws on other open libraries

## Components of Payment System Model

- Federal Reserve (RTGS)
- Funds Market
- Banks
- “Treasuries”
- Implement specific decision rules
- May learn via interaction with ...
- Treasury Adaptor

## Structure Supports Diverse Models of Decision Making

## Strategies

- Adaptive strategies (learning takes place)
- GENETICBANK is a bank learning through the process of a genetic algorithm
- CLASSIFIERBANK is a bank learning through a classifier system
- HEURISTICBANK is a bank that follows the heuristic rules described
- Static reference strategies (no learning)
- DELAYBANK is a bank following a pure strategy of delaying all payments and settling them at the end of the day (with end-of-day funding/defunding)
- ODBANK is a bank that follows the pure strategy of settling all payments immediately (with end-of-day funding/defunding)
- TITFORTATBANK is a bank that sends its first payment immediately and always delays subsequent payments until the time it receives funds (with end-of-day funding/defunding).

## Hypotheses

- Adaptive banks become better over time
- i.e. learning actually takes place. Successive iterations reduce total costs of settlement for a system consisting of adaptive banks of a type
- Adaptive banks become good in a homogenous environment
- a system consisting of trained adaptive banks of a type has lower average total costs than systems consisting of reference banks
- Adaptive banks become good in a mixed environment
- in a system consisting of adaptive banks of a type and reference banks of any type, adaptive banks become better over time and better than the reference banks

## Heuristic Bank Decision making

## Rules for settlement

- Banks settle arriving payments immediately if balance is above line D1-D2 and no payments are in queue
- Banks settle queued payments in FIFO order if balance is above line D1-D2
- Banks place arriving payments at the end of the queue if balance is below line D1-D2

## Borrowing and lending

- Rules for borrowing and lending
- banks post a bid to borrow if balance is below line B1-B2
- banks post an offer to lend if balance is above line L1-L2
- the amount posted is |balance-threshold| rounded up to the next million
- once a bid or offer is made, the bank cannot participate in the market for a given time-interval*
- banks withdraw all unmatched bids and offers if a payment arrives first (and make a new decision as above)
- Initially bids and offers are given on a fixed interest rate
- Subsequently
- The price will be something the banks learn and adapt to
- Bids and offers will be matched to form a payment or a series of payments
- Unmatched bids and offers will stay on the board until matched or withdrawn

- * to prevent too many transactions and at the same time allow for continuous decision making

## Cost Components

- Delay - proportional to time between arrival and execution using an implicit interest rate that reflects customer displeasure
- Intraday Overdraft - charged continuously at a specified rate
- Failure - charged at a specified rate for all payments remaining at the close
- Overnight Overdraft - charged at a specified rate for any negative balance
- Borrowing - paid at a specified funds rate plus a spread and a fixed transaction cost
- Lending - received at a specified funds rate minus a spread plus a fixed transaction cost

## Costs and remedies

- L1

- L2

- B1

- B2

- D1

- D2

- intraday

- overnight CB

- overnight CB

- overnight

- funds transaction

- Cost

- Parameter

- reduce value

- increase value

**Only effective if lending occurred****Only effective if borrowing occurred****Only effective if payments were delayed**

- The direction a parameter should be moved in order to decrease a cost

- delays

- borrowing

- intraday CB

- overnight market

- overnight market

- lending

## Adaptations - Parameters

- Banks adjust decision parameters based on the expected effect on the various costs that the parameter influences
- Directions are not sufficient: the expected magnitude of cost changes are needed as well
- Magnitudes are a linear approximation of the cost surface, which should be valid near the operating point
- Decomposing total cost and defining conditional parameter relevance in the costs-remedies table helps deal with some of the more pronounced non-linearities

## Adaptations – Coefficients

- Weights in the remedies table estimate the expected cost change for a unit change in the parameter
- Parameters are adjusted by:
- Finding the estimated change in total cost associated with increasing and decreasing each parameter
- Selecting the parameter and direction that appears to maximize cost reduction
- Updating the weights using the actual cost change observed for each component
- Uncertainty in the cost change can be included by
- Tracking the variance of the prediction error, and simulating a possible gradient value for each cost component, or
- Occasionally taking a random step

## Adaptive Process

***Ranges of***
***Cost***
***Reduction***

***Possible***
*** ******Cost***
***Reduction***

***Best ***
***Parameter***
***Move***

***Observe***
***Effect On***
***Cost***

*Change *
*Parameter*

*Calculate Total*
*Change*

*Sample*
*Coefficients*

*Update Coefficients*
*and Range*

## Results

- Simple system with:
- 9 banks
- 1500 payments per bank per day
- Lognormal payment size, mean = 1, sigma = 1

| Funding Mechanism | Rate | Period |
| --- | --- | --- |
| Daylight overdraft | 0.36% | Duration of overdraft |
| Overnight overdraft | 5% | 24 hours |
| Delay | 0.18% | Duration of delay |
| Failure | 6% | 24 hours |
| Federal funds | 4.5%+transaction fee | 24 hours |

- Comparison of reflexive strategies with adaptation
- Comparison of adapted strategies across banks

## Performance of Reflexive Strategies

|  |  | Contributors |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Strategy | Total Cost | Delay | Intraday OD | Transaction | Funding |
| Delay | 0.0018
(0.0022) | 21% | <1% | 7% | 72% |
| Pay | 0.0014
(0.0022)
 | <<1% | 2% | 8% | 90% |
| Tit-for-Tat | 0.0018
(0.0022) | 20% | <1% | 7% | 73% |

- Percentages

## Adaptation without Gradient Updating

- Example cost trajectories
- Example parameter trajectories

## Modifications and Refinements

- Heuristic model produced unexpected and counterintuitive behavior, including extensive borrowing and lending by the same bank in the same day, nonconvergent parameter values, and bursts of poor performance after quiescent periods
- We have implemented a succession of refinements to help impose more reasonable behavior, including
- Elaborating the cost components to insure monotonicity
- Completing the remediation matrix to include side effects
- Constraining parameter moves so that cost effects can be better inferred
- Imposing spreads and transaction costs to deter erratic funding
- These refinements improved performance however the behavior is still not completely “rational”

## Good Results for A Single Learner Costs

## Good Results for A Single Learner Parameters

## Good Results for A Single Learner Funding

## Limitations on Adaptation

- Nothing but balance governs decisions
- Response size is fixed and does not depend on cost gradient
- Uncertain environment rich with feedbacks; effects of parameter changes are difficult to discern amid the noise
- Response based on local sensitivities
- Learn on recent experience but forget past

## Adaptation with Gradient Updating

**Notes:**
Spike at .89 - > Kind of gridlock.  Several banks make moves to borrow less or lend more, and massive failures occur at all banks.  After two steps the moves are effectively retracted.

## Final Cost Distributions

## Comparison of Adapted Strategies

**Notes:**
Best bank is a net receiver, and lends everything to settle abruptly at the end of the day
Worst bank is a net sender, and incurs all of it’s costs from borrowing

## Hypotheses Revisited

- Adaptive banks become better over time
- Usually, but usually not permanently.
- Adaptive banks become good in a homogenous environment
- Some appear better and some are worse than the reference reflexive strategies.  We are interested in finding out whether some are worse *because* some are better.
- Adaptive banks become good in a mixed environment
- Still looking at this.  Against prompt banks, some can learn to make a profit, but can later forget this skill

## Preliminary Conclusions

- Cost matrix must be complete and responses should be monotonic, considering all side effects.  Deficiencies will be discovered and exploited
- Gradient following is unlikely to lead to a good solution
- Simultaneous parameter changes (e.g. raising L2 and lowering B2) may be needed to reduce costs.  The current implementation cannot discover these moves
- Cost function strongly depends on behavior of correspondents
- Current balance information alone may not be enough to inform a cost-minimizing decision
- A more robust search is likely to perform better.  Neural networks are appealing because they can shift among modes, and this strategy complements other adaptive methods we have implemented

## Next Ideas for Heuristic

- Distinguish counterparties and provide for performance awareness
- Reparameterize in terms of average balance and tolerance
- Revisit multiple parameter changes in a single step
- Slow parameter adjustment to provide a better estimate of consequences
- Constrain parameter ranges to exclude irrational combinations, such as funding-dominated solutions
- Allow concurrent payment and funding actions when both may be taken
- Adapt parameter change size
- Implement robust learning techniques

## Further Ahead

- Include simple funds market
- Evaluation of less intuitive decision formulations (genetic algorithm, classifier system, etc.)

## Spares

## Comparison with Reflexive Strategies

- Pay

- Delay

- Tit-for-Tat