# On a Unique Solution of the Stochastic Functional Equation Arising in Gambling Theory and Human Learning Process

Authors: Ali Turab; Awad A. Bakery; O. M. Kalthum S. K. Mohamed; Wajahat Ali
Journal: Journal of Function Spaces
 (2022)

---

## Abstract

The term “learning” is often used to refer to a generally stable behavioral change resulting from practice. However, it is a fundamental biological capacity far more developed in humans than in other living beings. In an animal or human being, the learning phase may often be viewed as a series of choices between multiple possible reactions. Here, we analyze a specific type of human learning process related to gambling in which a subject inserts a poker chip to operate a two-armed bandit device and then presses one of the two keys. Through the use of an electromagnet, one or more poker chips are given to the individual in a container located in the apparatus’s center. If a chip is provided, it is declared a winner; otherwise, it is considered a loser. The goal of this paper is to look at the subject’s actions in such situations and provide a mathematical model that is appropriate for it. The existence of a unique solution to the suggested human learning model is examined using relevant fixed point results.


---

## Body

## 1. Introduction

Learning is a fundamental biological capacity that is much more evolved in humans than in any other living being. The central topic in learning philosophy is how multiple forms of learning take place in a human brain and body since this was explicitly formulated in the discipline of learning psychology, but with additional feedback from other psychological disciplines and the adjacent areas of sociology, pedagogy, and biology, including contemporary brain science.In modern mathematical learning experiments, the researchers concluded that a basic learning experiment was compatible with any stochastic process. Thus, it is not a novel concept (for detail, see [1]). However, after 1950, two critical features emerged mainly in the research initiated by Bush, Estes, and Mosteller. Firstly, the learning method egalitarian essence was a core feature of the developed model. Secondly, these frameworks were studied and applied in areas that did not conceal their quantitative aspects.Several studies on human actions in probability-learning scenarios have produced different results (for the detail, see [2–5]).In 2019, Turab and Sintunavarat [6, 7] proposed a functional equation to examine the experimental work of Bush and Wilson [8] on a paradise fish. In this experiment, a fish was given two options for swimming. The fish had options to swim on either side (right or left) of the tank’s far end.In [9], the authors recently addressed a kind of traumatic avoidance learning experiment for normal dogs suggested by Solomon and Wynne [10]. They examined the psychological responses of 30 dogs enclosed in a small steel grid cage and proposed a mathematical model. The suggested avoidance learning model’s existence and uniqueness of a solution result were investigated using the appropriate fixed point method.For the research in this area, especially related to the two-choice behavior, we refer to [11–13] and the references therein. It is worth noting that most animal behavior studies in a two-choice situation discussed above have focused only on the animals’ approach toward an inevitable conclusion. Bush and Wilson [8], on the other hand, divided such responses into four categories depending on the food source and side chosen (right-reward, right nonreward, left reward, and left nonreward).In this work, by following the work presented by Turab and Sintunavarat [6, 9] and the idea discussed in [8, 14], our aims are to discuss the two-armed bandit experiment proposed by Goodnow and Pettigrew [15] and propose a convenient mathematical model. We evaluate our findings under the experimenter-subject controlled events to see the feasibility of the suggested model. The existence of a unique solution to the proposed model is examined by using the appropriate fixed point theorem. In the end, we raise some open problems for the interested readers.


## 2. A Two-Armed Bandit Experiment

In [15], Goodnow and Pettigrew presented an experiment related to the gambling theory. This gambling activity involves playing a poker game with chips worth one penny each (see Figure 1). The subject (S) is given 200 chips by an experimenter (E). He/She inserts into the machine one of these chips and pushes one of two buttons. A chip drops into the payout box with a clatter of noise when the bet is successful. The payoff box has a glass face, and the heap of chips he/she has won can be seen by S. The subject is not permitted until the end of the experiment to carry the chips out of this box. Whatever the outcome of the bet, between each test, the machine becomes unusable for several seconds, and S wait until two signal lights and a loud buzz appear, indicating that the device is ready to take the next bet. The apparatus is fully programmed such that inserting a chip before the device’s ready is useless for S.Figure 1
A sketch of a two-armed bandit machine.When the subjectS implants a chip (upper center light) and clicks a key (left or right lower), the lights on the face of the machine flash on successively (upper outer lights in Figure 1). These lights are parallel to the control machine’s lights controlled in an adjacent space by E. A master switch to turn the device on or off is also included in the control machine, along with a key that allows the machine to eject a chip into the pay-off box when pushed. The one-way mirror enables E from the control room to view S’s activities.


### 2.1. Procedure

The assignment’s method and directions were given toS and E. The S was instructed that he/she is playing for cash and that he/she would be paid for the discrepancy between the number of wins and losses. There were 120 trials allowed for every S, divided into 12 blocks of 10 trials each. The probability of the above task was 50 : 50, 70 : 30, and 90 : 10. When the experiment is completed, S was asked the following questions:
(1)
How did you decide which alternative you should choose?(2)
How he/she thought about the strategy of always betting on one key?


### 2.2. Results

The results were described in terms of the average proportion of choices of one alternative: pushing the ‘left button’ in the gambling experiment provided the greater likelihood of these alternatives outside the 50 : 50 scenario. In Table1, the findings are presented.Table 1
Mean proportional choices by group and by blocks of 20 trials of pressing a ‘left key’ in gambling experiment.
GroupTrials50 : 5070 : 3090 : 101-200.4300.4890.76521-400.5050.6640.87841-600.5500.7210.95061-800.4950.7220.95481-1000.4650.7820.965101-1200.5150.8150.964N1014∗14∗∗Those two groups where N was increased to 14 because the data were required for another purpose (an analysis of choice sequences), and the incidence of 100 : 0 choice distributions was cutting down on the amount of data available for such analysis.


## 2.1. Procedure

The assignment’s method and directions were given toS and E. The S was instructed that he/she is playing for cash and that he/she would be paid for the discrepancy between the number of wins and losses. There were 120 trials allowed for every S, divided into 12 blocks of 10 trials each. The probability of the above task was 50 : 50, 70 : 30, and 90 : 10. When the experiment is completed, S was asked the following questions:
(1)
How did you decide which alternative you should choose?(2)
How he/she thought about the strategy of always betting on one key?


## 2.2. Results

The results were described in terms of the average proportion of choices of one alternative: pushing the ‘left button’ in the gambling experiment provided the greater likelihood of these alternatives outside the 50 : 50 scenario. In Table1, the findings are presented.Table 1
Mean proportional choices by group and by blocks of 20 trials of pressing a ‘left key’ in gambling experiment.
GroupTrials50 : 5070 : 3090 : 101-200.4300.4890.76521-400.5050.6640.87841-600.5500.7210.95061-800.4950.7220.95481-1000.4650.7820.965101-1200.5150.8150.964N1014∗14∗∗Those two groups where N was increased to 14 because the data were required for another purpose (an analysis of choice sequences), and the incidence of 100 : 0 choice distributions was cutting down on the amount of data available for such analysis.


## 3. Mathematical Modeling of the Two-Armed Bandit Experiment

In the above experiment, significant interest lies in the behavior of a subjectS; press right or left button, `A1’ or `A2,’ and get the reward in terms of a poker chip. In our view, if a subject chooses the reward side, there would be an occurrence of alternative O1, and if the subject made a move to the other side, then there will be an occurrence of alternative O2. Thus, according to the mathematical point of view, there would be four possibilities of events, depending on the action of the subject and the reward. These events are listed in Table 2.Table 2
The possible four responses in two-armed bandit experiment.
ResponsesOutcomesA1: press right buttonO1: reward (poker chips)A1: press right buttonO2: no reward (no poker chips)A2: press left buttonO1: reward (poker chips)A2: press left buttonO2: no reward (no poker chips)Depending on the action of the subject and getting the chance of the reward, we have the following four events (see Table3).Table 3
The corresponding events of the subject.
ResponseOutcomesEventsA1O1E1A1O2E2A2O1E3A2O2E4The probability of the outcomesA1 and A2 are x and 1−x, respectively, where x∈0,1]. The experimental pattern asks for the outcomes of the responses (whether the subject get the reward or not), trials’ fixed proportion of p∈0,1]. Therefore, we get the event probabilities stated below (see Table 4).Table 4
Probabilities of the four events.
EventProbability of occurrenceE1pxE21−pxE3p1−xE41−p1−xWe defineη1,η2,η3,η4∈0,1 as the learning rate parameters and their values can be recognized as a measure of the ineffectiveness of the corresponding events E1−E4 in altering the response probability.If, on some trial,px is the possibility of response A1 with outcome O1 and A1 is fulfilled, the next possibility of A1 with outcome O1 will be η1x+1−η1, and if A1 is achieved with outcome O2 then the new probability would be η2x+1−η2 with the event probability 1−px. Similarly, if A2 is performed with outcomes O1 and O2, then the new probabilities of A2 are η3x+1−η3 and η4x+1−η4, with the event probabilities p1−x and 1−p1−x, respectively. For the four events E1−E4, we can define the transition operators Q1−Q4:0,1⟶0,1] as
(1)Q1x=η1x+1−η1,Q2x=η2x+1−η2,Q3x=η3x+1−η3,Q4x=η4x+1−η4,for all x∈0,1].By considering the work presented in [6, 8, 9] and the above transition operators with their corresponding probabilities and events given in Table 4, we introduce the following functional equation, which can discuss all the aspects of the two-armed bandit model.
(2)Qx,η1,η2,η3,η4=pxQη1x+1−η1,η1+1−pxQη2x+1−η2,η2+p1−xQη3x+1−η3,η3+1−p1−xQη4x+1−η4,η4.Fixed point theory, on the other hand, began in the second half of the nineteenth century as a method of using iterative estimations to demonstrate the existence and uniqueness of solutions to ordinary differential and integral equations. It is a wonderful combination of basic and applied analysis, geometry, and topology. A fixed point theoretic viewpoint can be seen in Picard’s work, which is a fundamental notion in the field of metric fixed point theory. Nevertheless, it is credited to the Polish mathematician “Banach,” who abstracted the underlying principles into a framework that can be applied to find the existence of a unique solution to the broad range of applications beyond differential and integral equations. It has been extended and generalized in numerous directions (for the detail, see [16–18]). We suggest the reader to see [19–21] for further information on fixed point theory and its applications in various spaces.The following stated outcome will be required in the progression.Theorem 1 (see [22]).
LetO,d be a complete metric space and J:O⟶O be a Banach contraction mapping (shortly, BCM), that is,
(3)dJω,Jϖ≤δdω,ϖ,for some δ<1 and for all ω,ϖ∈O. Then, O has one fixed point. Furthermore, the Picard iteration ωn in O that can be defined as ωn=Oωn−1 for all n∈ℕ, where ω0∈O, converges to the unique fixed point of O.


## 4. Existence and Uniqueness Results

We letO=0,1. For the rest of this article, D represents the class J:O⟶ℝ with J0=0 consisting of all real-valued continuous functions which satisfy the following relation
(4)supω≠ϖJω−Jϖω−ϖ<∞.Clearly,D,· is a Banach space with
(5)J=supω≠ϖJω−Jϖω−ϖ,for all J∈D.Following that, we can rewrite the functional equation (2) as
(6)Jx=pxJη1x+1−η1+1−pxJη2x+1−η2+1−xpJη3x+1−η3+1−p1−xJη4x+1−η4,where J:O⟶ℝ is an unknown function, 0<η1,η2,η3,η4<1.Theorem 2.
For0<η1,η2,η3,η4<1 and p∈O with Θ1<1, where
(7)Θ1≔2pη1+η3+21−pη2+η4+2p.
If there is aC⊆D such that C is W-invariant, that is, WC⊆C, where W:C⟶C is defined for each J∈C as
(8)WJx=pxJη1x+1−η1+1−pxJη2x+1−η2+1−xpJη3x+1−η3+1−p1−xJη4x+1−η4,for all x∈O, then W is a BCM.Proof.
LetJ1,J2∈C. For each distinct points ω,ϖ∈O, we obtain
(9)WJ1−WJ2ω−WJ1−WJ2ϖ∣ω−ϖ∣=1ω−ϖpωJ1−J2η1ω+1−η1+1−pωJ1−J2η2ω+1−η2+p1−ωJ1−J2η3ω+1−η3+1−p1−ωJ1−J2η4ω+1−η4−pϖJ1−J2η1ϖ+1−η1−1−pϖJ1−J2η2ϖ+1−η2−p1−ϖJ1−J2η3ϖ+1−η3−1−p1−ϖJ1−J2η4ϖ+1−η4=1ω−ϖpωJ1−J2η1ω+1−η1−pωJ1−J2η1ϖ+1−η1+1−pωJ1−J2η2ω+1−η2−1−pωJ1−J2η2ϖ+1−η2+p1−ωJ1−J2η3ω+1−η3−p1−ωJ1−J2η3ϖ+1−η3+1−p1−ωJ1−J2η4ω+1−η4−1−p1−ωJ1−J2η4ϖ+1−η4+pωJ1−J2η1ϖ+1−η1−pϖZ1−J2η1ϖ+1−η1+1−pωJ1−J2η2ϖ+1−η2−1−pϖJ1−J2η2ϖ+1−η2+p1−ωJ1−J2η3ϖ+1−η3−p1−ϖJ1−J2η3ϖ+1−η3+1−p1−ωJ1−J2η4ϖ+1−η4−1−p1−ϖJ1−J2η4ϖ+1−η4=1ω−ϖpωJ1−J2η1ω+1−η1−pωJ1−J2η1ϖ+1−η1+1ω−ϖ1−pωJ1−J2η2ω+1−η2−1−pωJ1−J2η2ϖ+1−η2+1ω−ϖp1−ωJ1−J2η3ω+1−η3−p1−ωJ1−J2η3ϖ+1−η3+1ω−ϖ1−p1−ωJ1−J2η4ω+1−η4−1−p1−ωJ1−J2η4ϖ+1−η4+1ω−ϖpωJ1−J2η1ϖ+1−η1−pϖJ1−J2η1ϖ+1−η1+1ω−ϖ1−pωJ1−J2η2ϖ+1−η2−1−pϖJ1−J2η2ϖ+1−η2+1ω−ϖp1−ωJ1−J2η3ϖ+1−η3−p1−ϖJ1−J2η3ϖ+1−η3+1ω−ϖ1−p1−ωJ1−J2η4ϖ+1−η4−1−p1−ϖJ1−J2η4ϖ+1−η4≤pωJ1−J2η1ω+1−η1−pωJ1−J2η1ϖ+1−η1η1ω−η1ϖ×η1ω−η1ϖω−ϖ+1−pωJ1−J2η2ω+1−η2−1−pωJ1−J2η2ϖ+1−η2η2ω−η2ϖ×η2ω−η2ϖω−ϖ+p1−ωJ1−J2η3ω+1−η3−p1−ωJ1−J2η3ϖ+1−η3η3ω−η3ϖ×η3ω−η3ϖω−ϖ+1−p1−ωJ1−J2η4ω+1−η4−1−p1−ωJ1−J2η4ϖ+1−η4η4ω−η4ϖ×η4ω−η4ϖω−ϖ+pJ1−J2η1ϖ+1−η1+1−pJ1−J2η2ϖ+1−η2+pJ1−J2η3ϖ+1−η3+1−pJ1−J2η4ϖ+1−η4.
By applying the definition of the norm (5), we obtain
(10)WJ1−WJ2ω−WJ1−WJ2ϖω−ϖ≤η1pωJ1−J2+η21−pωJ1−J2+η3p1−ωJ1−J2+η41−p1−ωJ1−J2+pJ1−J2η1ϖ+1−η1−pJ1−J20+1−pJ1−J2η2ϖ+1−η2−1−pJ1−J21+pJ1−J2η3ϖ+1−η3−pJ1−J20+1−pJ1−J2η4ϖ+1−η4−1−pJ1−J21=η1pωJ1−J2+η21−pωJ1−J2+η3p1−ωJ1−J2+η41−p1−ωJ1−J2+pη1ϖ+1−η1J1−J2+1−pη2ϖ−η2W1−J2+pη3ϖ+1−η3J1−J2+1−pη4ϖ−η4J1−J2≤Θ1J1−J2,where Θ1 is defined in (7). This gives that
(11)dWJ1,WJ2=WJ1−WJ2≤Θ1J1−J2=Θ1dJ1,J2.
As a result of0<Θ1<1, we can claim that W is a BCM with the metric d imposed by ·.
We get the following conclusion from Theorem2 about the uniqueness of a functional equation (6)’s solution.☐☐Theorem 3.
The stochastic equation (6) has a unique solution with Θ1<1, where Θ1 is defined in (7). Assume that there is a C⊆D such that C is W-invariant, that is, WC⊆C, where W:C⟶C defined for each J∈C as
(12)WJx=pxJη1x+1−η1+1−pxJη2x+1−η2+p1−xJη3x+1−η3+1−p1−xJη4x+1−η4,for all x∈O. Furthermore, the following iteration Jn in C∀n∈ℕandJ0∈C defined by
(13)Jnx=pxJn−1η1x+1−η1+1−pxJn−1η2x+1−η2+p1−xJn−1η3x+1−η3+1−p1−xJn−1η4x+1−η4,
converges to the unique solution of (12).Proof.
We reach the conclusion of this theorem by combining the Banach fixed point theorem with Theorem2.☐The following corollaries arise from the preceding findings.Corollary 4.
For0<η1≤η2≤η3≤η4<1 and p∈O with Θ~1<1, where
(14)Θ~1≔2p+η4.
If there is aC⊆D such that C is W-invariant, that is, WC⊆C, where W:C⟶C defined for each J∈C as
(15)WJx=pxJη1x+1−η1+1−pxJη2x+1−η2+1−xpJη3x+1−η3+1−p1−xJη4x+1−η4,for all x∈O, then W is a BCM.Corollary 5.
The stochastic equation (6) has a unique solution with Θ~1<1, where Θ~1 is defined in (7). Assume that there is a C⊆D such that C is W-invariant, that is, WC⊆C, where W:C⟶C defined for each J∈C as
(16)WJx=pxJη1x+1−η1+1−pxJη2x+1−η2+p1−xJη3x+1−η3+1−p1−xJη4x+1−η4,for all x∈O. Furthermore, the iteration Jn in C (∀n∈ℕandJ0∈C) defined by
(17)Jnx=pxJn−1η1x+1−η1+1−pxJn−1η2x+1−η2+p1−xJn−1η3x+1−η3+1−p1−xJn−1η4x+1−η4,converges to the unique solution of (12).


## 5. A Certain Case with Experimenter-Subject-Controlled Events

It has been highlighted that the examination of any experiment is truly based on suppositions. Therefore, experiments are classified into contingent and noncontingent, based on the occurrences of the results. It has been suggested that the correspondence of contingent experiments is for the events of experimental-subject (contingent) and noncontingent experiments are for the events of experimental control.In the previous models on imitation problems such as T-maze experiments with fish and dog (see [6, 9]), it was already mentioned that such experiments required a contingent approach; the result of the trials was entirely dependent on the subject’s choice. Thus, such types of models required experimenter-subject-controlled events. The two responses A1 and A2, along with outcomes O1 and O2, are choosing the right or left side or pushing the right or left button, which coincides with rewarding and non-rewarding or correct and incorrect, respectively. Now we define the probabilities p1 and p2 which indicate the conditional probability of outcomes O1 and O2 of the given alternatives A1 and A2, respectively. With such conditions, we have the following Table 5.Table 5
Four events under conditional probability of occurrence.
EventsOutcomesTransition operatorsProbabilities of occurrenceA1O1Q1x=η1x+1−η1p1xA1O2Q2x=η2x+1−η21−p1xA2O1Q3x=η3x+1−η3p21−xA2O2Q4x=η4x+1−η41−p21−xWe have the following functional equation from the data given above:(18)Jx=p1xJη1x+1−η1+1−p1xJη2x+1−η2+p21−xJη3x+1−η3+1−p21−xJη4x+1−η4,where J:O⟶ℝ is an unknown function, 0<η1,η2,η3,η4<1 and p1,p2∈O. We shall begin with the following finding.Theorem 6.
For0<η1,η2,η3,η4<1 and p1,p2∈O with Θ2<1, where
(19)Θ2≔2p1η1−η2+2p2η3−η4+2η2+η4+p1+p2.
Assume that, if there is aC⊆D such that C is W-invariant, that is, WC⊆C, where W:C⟶C defined for each J∈C as
(20)WJx=p1xJη1x+1−η1+1−p11−xJη2x+1−η2+p21−xJη3x+1−η3+1−p21−xJη4x+1−η4,for all x∈O, then W is a BCM.Proof.
LetJ1,J2∈C. For each distinct points ω,ϖ∈O, we obtain
(21)∣WJ1−WJ2ω−WJ1−WW2ϖ∣∣ω−ϖ∣=1ω−ϖp1ωJ1−J2η1ω+1−η1+1−p1ωJ1−J2η2ω+1−η2+p21−ωJ1−J2η3ω+1−η3+1−p21−ωJ1−J2η4ω+1−η4−p1ϖJ1−J2η1ϖ+1−η1−1−p1ϖJ1−J2η2ϖ+1−η2−p21−ϖJ1−J2η3ϖ+1−η3−1−p21−ϖJ1−J2η4ϖ+1−η4=∣1ω−ϖp1ωJ1−J2η1ω+1−η1−p1ωJ1−J2η1ϖ+1−η1+1−p1ωJ1−J2η2ω+1−η2−1−p1ωJ1−J2η2ϖ+1−η2+p21−ωJ1−J2η3ω+1−η3−p21−ωJ1−J2η3ϖ+1−η3+1−p21−ωJ1−J2η4ω+1−η4−1−p21−ωJ1−J2η4ϖ+1−η4+p1ωJ1−J2η1ϖ+1−η1−p1ϖJ1−J2η1ϖ+1−η1+1−p1ωJ1−J2η2ϖ+1−η2−1−p1ϖJ1−J2η2ϖ+1−η2+p21−ωJ1−J2η3ϖ+1−η3−p21−ϖJ1−J2η3ϖ+1−η3+1−p21−ωJ1−J2η4ϖ+1−η4−1−p21−ϖJ1−J2η4ϖ+1−η4∣=1ω−ϖp1ωJ1−J2η1ω+1−η1−p1ωJ1−J2η1ϖ+1−η1+1ω−ϖ1−p1ωJ1−J2η2ω+1−η2−1−p1ωJ1−J2η2ϖ+1−η2+1ω−ϖp21−ωJ1−J2η3ω+1−η3−p21−ωJ1−J2η3ϖ+1−η3+1ω−ϖ1−p21−ωJ1−J2η4ω+1−η4−1−p21−ωJ1−J2η4ϖ+1−η4+1ω−ϖp1ωJ1−J2η1ϖ+1−η1−p1ϖJ1−J2η1ϖ+1−η1+1ω−ϖ1−p1ωJ1−J2η2ϖ+1−η2−1−p1ϖJ1−J2η2ϖ+1−η2+1ω−ϖp21−ωJ1−J2η3ϖ+1−η3−p21−ϖJ1−J2η3ϖ+1−η3+1ω−ϖ1−p21−ωJ1−J2η4ϖ+1−η4−1−p21−ϖJ1−J2η4ϖ+1−η4≤p1ωJ1−J2η1ω+1−η1−p1ωJ1−J2η1ϖ+1−η1η1ω−η1ϖ×η1ω−η1ϖω−ϖ+1−p1ωJ1−J2η2ω+1−η2−1−p1ωJ1−J2η2ϖ+1−η2η2ω−η2ϖ×η2ω−η2ϖω−ϖ+p21−ωJ1−J2η3ω+1−η3−p21−ωJ1−J2η3ϖ+1−η3η3ω−η3ϖ×η3ω−η3ϖω−ϖ+1−p21−ωJ1−J2η4ω+1−η4−1−p21−ωJ1−J2η4ϖ+1−η4η4ω−η4ϖ×η4ω−η4ϖω−ϖ+p1J1−J2η1ϖ+1−η1+1−p1J1−J2η2ϖ+1−η2+p2J1−J2η3ϖ+1−η3+1−p2J1−J2η4ϖ+1−η4.
By applying the definition of the norm (5), we obtain
(22)WJ1−WJ2ω−WJ1−WJ2ϖω−ϖ≤η1p1ωJ1−J2+η21−p1ωJ1−J2+η3p21−ωJ1−J2+η41−p21−ωJ1−J2+p1J1−J2η1ϖ+1−η1−p1J1−J20+1−p1J1−J2η2ϖ+1−η2−1−p1J1−J21+p2J1−J2η3ϖ+1−η3−p2J1−J20+1−p2J1−J2η4ϖ+1−η4−1−p2J1−J21=η1p1ωJ1−J2+η21−p1ωJ1−J2+η3p21−ωJ1−J2+η41−p21−ωJ1−J2+p1η1ϖ+1−η1J1−J2+1−p1η2ϖ+1−η2J1−J2+p2η3ϖ+1−η3J1−J2+1−p2η4ϖ+1−η4J1−J2≤Θ2J1−J2,where Θ2 is defined in (19). Thus, we have
(23)dWJ1,WJ2=WJ1−WJ2≤Θ2J1−J2=Θ2dJ1,J2.
As a result of0<Θ2<1, one can see that W is a BCM.☐☐For the unique solution of (18), we get the subsequent conclusion from Theorem 6.Theorem 7.
The stochastic equation (18) has a unique solution with Θ2<1. Assume that, there is a C⊆D such that C is W-invariant, that is, WC⊆C, where W:C⟶C defined for each J∈Cas(24)WJx=p1xJη1x+1−η1+1−p11−xJη2x+1−η2+p21−xJη3x+1−η3+1−p21−xJη4x+1−η4,for all x∈O. Furthermore, the iteration Jn in C∀n∈ℕandJ0∈C defined by
(25)Jnx=p1xJn−1η1x+1−η1+1−p11−xJn−1η2x+1−η2+p21−xJn−1η3x+1−η3+1−p21−xJn−1η4x+1−η4,converges to the unique solution of (24).Proof.
The conclusion of this theorem can be found by combining Theorem6 with the Banach fixed point theorem.☐☐


## 6. Conclusion

In this work, we have discussed a special type of stochastic process related to the two-armed bandit experiment [15] which plays a vital role in observing the subject’s behavior in a two-choice situation. We reviewed the operant’s responses under such conditions and provided a mathematical model for it. The Banach fixed point theorem was used to determine the existence of a unique solution to the two-armed bandit learning model. We investigated the proposed model’s adaptability by subjecting it to some controlled events. Moreover, the presented approach is straightforward and easy to verifiable. Thus, the proposed approach can be used to investigate more psychological learning experiments related to animals and humans in the future.Now, for the interested readers, we propose the following open problems.Question 1.
Assume that if a subject does not press any button on a specific trialk, how can we describe such an event by a model?In the end, we also leave the stability problem (for the detail, see [23–27]) of the stochastic equation given below as an open problem:
(26)Jx=pxJη1x+1−η1+1−pxJη2x+1−η2+p1−xJη3x+1−η3+1−p1−xJη4x+1−η4,where 0<η1,η2,η3,η4<1 and J:O⟶ℝ is an unknown function.



---

*Source: 1064803-2022-01-20.xml*