# JSn 

# tbe Wniteb**~tates** 

# Qeourt ofjf 

# eberal Qelailns No. 14-67C (Filed: June 30, 2017) (Re-filed: August 3, 2017) 1 

### **************************** KATHLEEN KAPLAN,

Plaintiff, 

# FILED AUG
.32017

OSM U

.S

. COURT OF FEDERAL CLAIMS

v. Equal Pa y Act; 29 U.S.C

. § 206(d)(l); Merit system.
THE UNITED STATES, Defendant. 

### **************************** Kathleen M Kaplan, Arlington, VA,pro se. DanielK. Greene, United StatesDepartmentofJustice, CivilDivision, Commercial Litigation Branch, Washington, DC, with whom were ChadA. Readier, Acting Assistant Attorney General, Robert 

### E. Kirschman, Jr., Director, ReginaldT Blades, Jr., AssistantDirector, and DeborahA. Bynum, Assistant Director for defendant.

OPINION BRUGGINK, Judge. Plaintiff,Dr. KathleenKaplan, appearingpro se, filed hercomplainton January 27, 2014

. Plaintiff is a former employee ofthe Air Force Office of Scientific Research ("AFOSR"), which is part of the Air Force Research Laboratory("AFRL"). Shebrings this suitundertheEqual PayAct. 29 U.S.C.

1This opinionwas originallyissued undersealpursuanttotheprotectiveorder entered in this case. Defendant timely offered proposed redactions, but plaintiff did not submit any proposed redactions to chambers. We adopt defendant's proposed redactions because we find them to be appropriate. Those redactions are indicated herein with brackets.§ 206(d)( 1) (2012). She contends thatwhile shewas employed atAFOSR, she was paid less than males doing comparable work. She claims that she is entitled to additional pay beginning in 2011. We held a three-day trial from December 6to December 8, 2016. Afterpost-trialbriefing, the matter is ready for disposition. Becausethe governmenthas demonstratedthat ithad in place a merit system that explains the differential in pay, the Laboratory Personnel Demonstration Project, we find against plaintiff. In reaching our finding in favor ofthe government, we are mindful that Dr. Kaplan appearsprose. BACKGROUND Dr. Kaplan is trained as a computer scientist. She earned three degrees in computer science: a B.S. from the University ofMassachusetts Lowell, a M.S. from the Florida Institute of Technology, and a D.Sc. from George Washington University. Trial Transcript ("Tr.") 46; PX 121. 2 Dr. Kaplan spent a number ofyears in academia before moving to AFOSR in August of 2005. I. The Air Force Office ofScientific Research The AFOSR, part ofthe Air Force Research Laboratory, manages the research investment for the United States Air Force. AFOSR's mission is to "[d]iscover, shape, and champion basic science that profoundly impacts the future Air Force." PX 145 at KAPL005102. In large part, the agencypursues its mission by directly funding research by others as well as through certain scholarship and outreachprograms. Overthe years, the agencyhas undergone several reorganizations. The most recent occurred in 2013 and resulted in AFOSRchangingfrom havingthree "directorates"-thePhysics & Electronics Directorate; the Mathematics, Information, & Life Sciences Directorate; and the Aerospace, Chemical & Material Sciences Directorate-to five "divisions," namely, Dynamical Systems & Control; Quantum & NonEquilibrium Processes; Information, Decision, & Complex Networks; Complex Material and Devices; and Energy, Power, & Propulsion. II. Dr. Kaplan and Her Alleged Comparators

2 "PX"refers to admitted exhibits offeredbyplaintiff; "DX"refers to admitted exhibits offered by defendant. 2During her tenure at AFOSR, she held three different positions. She began in 2005 as a program manager, which she held from herhire date until January of2011. Tr. 53. As the result ofan Equal Employment Opportunity settlement, Dr. Kaplan left her program manager position and became the deputy director ofthe Physics & Electronics Directorate on January 10, 2011. Tr. 67. Sheheldherdeputy directorpositionuntil AFOSRcompletedthe 2013 reorganization, which resulted in her assuming a program officer position in theInformation, Decision, & ComplexNetworks division in Februaryof2013. Plaintiff claims that she is entitled to an increase in pay based on her performance in these three positions. Because ofthe lapse oftime between performance, evaluation, and the time for which pay increases became effective, the relevantperformance assessment cycles began in October 2009. Dr. Kaplan offers sixmale**comparators~men**who she arguesreceived more pay while performing workthatrequired substantially equal skill, effort, andresponsibility. Heralleged comparators, two for eachposition, areDr. Wu and Dr. Schlossberg for her program manager position, Dr. DeLong and Dr. Blackwood for her deputy director position, and Dr. Bonneau and Dr. Nachman for her program officer position. Defendant does not concede that these men are proper comparators, even ifthey might have shared a title with plaintiff, becausetheirworkdidnotinvolve equal skill, effortorresponsibility. Defendant also argues that the agency used a merit-based system that linked pay to performance, thus providing a reason for the difference in pay regardless ofwhether the comparators' work involved equal skill, effort, or responsibility. In view ofthe importance of the agency's novel system of linkingpaytoperformance, knownas theLaboratoryPersonnelDemonstration Project, we begin with an extensive description ofthat program. III. The Laboratory Personnel Demonstration Project TheAFRLreplacedtheGeneral Schedule("GS") compensationsystem with the Laboratory Personnel Demonstration Project (the "Lab Demo") in March 1997, hence it was in place throughout plaintiffs employmentperiod and its effects wouldhave been reflected in the pay Dr. Kaplanreceived at the commencement ofthe years at issue here. Tr. 547. As described in AFRL Manual 36-104 (the "Manual"),3 the Lab Demo "is an improved personnel

3 Dr. Kaplan testified that she had access to the Manual and was aware ofthe Lab Demo system at all times relevant to this litigation. Tr. 157. 3management system which provides laboratory management, at the lowest practical level, the flexibility, authority, and control to achieve a quality workforce and laboratory." PX 50 at KAPL000238.4 In relevantpart, the Lab Demo replaced the GS 1-15 grade levels used commonly across federal civilian employment with four broadband levels, DR-I through DR-IV, with DR-IVbeing the highest grade and equivalent to GS-15. Id. at KAPL000241. Dr. Kaplan and each ofher alleged male comparators took part in what is known as the "DR pay plan," which is specific to AFRL scientists and engineers ("S&E"). Id. at KAPL000238. In order to "provide an effective, efficient, and flexible method for assessing, compensating, and managing the laboratory workforce," the Lab Demo implements whatis known as a contribution-basedcompensation system ("CCS"). Id. A contribution is the "measure ofthe value ofwhat an employee accomplished." DX 5 at AFKAP032215. As opposed to simplyjudging an employee's performance, in assessing an employee's contribution, the focus is onthepositiveimpact an employee's actions have with respectto advancing "thepurpose, vision, mission, and objectives ofthe organization,"here, theAir Force. Id. That is, the system seeks to "[n]ever confuse motion with progress," Tr. 838 (Dr. Christian), nor is mere longevity rewarded for its own sake. Every year, S&E employees are assessed and given scores by their supervisors based on four factors: problem solving, communication, technology management, and teamwork and leadership. Each factor has a corresponding descriptor-oftenreferred to as a "rubric"-thatdescribes the expected level of work specific to the S&E career path and an employee's particular broadband level. These rubrics are published in the AFRL Manual 36-104, to which the employees have access. PX 50 at KAP000353-56; Tr. 157. In a process described in more detail below, the four factor scores are averaged to determine an employee's Overall Contribution Score ("OCS"), which, in conjunction with the standard pay line ("SPL"),5 serves as the

4 The page citations for exhibits refer to the Bates pages provided during discovery.

5 The SPL relates an employee's contribution to basic pay for a given year. It is expressed on a graph with possible contribution scores along the x-axis and potential annual basicpay amounts alongthe y-axis. The SPL is a straight line with a positive slope and can be used to determine an employee's expected 4primary basis for individual pay increases beyond an employee's then-current pay. In theory, the CCS allows Lab Demo employees to more quickly reach the equivalent of a GS-15/ step-10 than under the GS system, if their contributions merit such a progression. There are also two types ofbonuses that are available to Lab Demo employees. The broadband IV bonus is used when an "employee's OCS indicates that he/she should receive a base pay increase that would place him/her over the GS-15/step-10 pay cap." 6 Id. at KAPL000310. What is referredto as the CCS bonus is used "torecognizeunsustainable contributions, [for example if! the employee had an opportunity in a given year that's not going to last

... they might be recognized with a CCS bonus." Tr. 579. This gives the employee's supervisors the ability to reward a one-offcontribution without increasing the employee's expected OCS for the next cycle. According to the Manual, "[t]he annual CCS assessment scoring process begins with input from the employee, which provides an opportunity to state the perceived accomplishments and level ofcontribution." PX 50 at KAPL000280-81. The assessment cycle begins on October 1 and ends on September 30 ofthe following year. Id. at KAPL000281. For example, an assessment cyclethat runs from October 1, 2016 to September 30, 2017 would determine an employee's pay for the 2018 calendar year. For that reason, the pay Dr. Kaplan received in January 2011, the beginning ofwhat she asserts is the relevant limitations period, was a reflection ofthe prior operation ofthe LabDemo from 2005through2009, insofaras herperformancewas evaluated. Atthe beginning ofthe cycle, supervisors are directed to "provide the broadband level descriptors, factors, and key elements to employees, as well as discuss expectations for the upcoming assessmentperiod so thatemployees !mowthe basis on whichtheir contributionwill be assessed." Id. TheManual also dictates that there be a mid-cycle review in April or May ofa particular cycle. "At this time, the employees' contributions to the mission will be
OCS, given her current salary, as well as to determine whether an employee is over-, under-, or equitably compensated, given her actual contribution. An employee is considered equitably compensated ifher pay is within 0.3 OCS above or below the SPL. PX 50 at KAPL000279.

6 We note that Dr. Kaplan never received a score high enough to warrant a broadband IV bonus. 5discussed, as well as future professional development and careeropportunities.

. . . The supervisor completes the AFRL Form 279, 

### CCS Feedback, to facilitate and document the mid-cycle review." Id. The scoring process occurs during the end-of-cycle assessment, which begins when the employee performs a self-assessment, summarizing their contributions in a way that "reflect[s] the impact or result of each activity ratherthanjustlistingthe activity itself." Id. These self-assessments are then reviewed by the employee's first-level supervisor as he or she creates a preliminary assessment, which "is determined using the employee input, to include any teamparticipation, and the supervisor'spersonal knowledge ofthe employee's overall contribution to the laboratory mission." Id. at KAPL000282. The first-level supervisor then takes the preliminary assessment to the First-LevelMeeting ofManagers ("First-Level MoM"), the opportunity for an employee's first-level supervisor and first-level supervisors of similar employeesto "meetwiththeir respective second-level supervisortoreview and discusspreliminaryassessments,refinetheminto numericalCCS factor scores, and adjust any factor write-ups based on results ofthe meeting." Id. As mentioned above, each factorhas a corresponding setofdescriptors, or a rubric, that describes the expected contribution specific to the four broadband levels. Supervisors are to use these rubrics in their evaluations. Factor scores can range from 0.0 to 5.9. Contributions that are assessed to con-espond with the descriptors associated with the level I broadband receive a score within the range of 0.75 to 1.9. "Contributions meeting level II descriptors are assessed at scores of 2.0 to 2.9, and so forth." Id. Contributions that fail to meetthe descriptors for a level I contributionreceive a score of0.0; similarly, contributions relevantto a specific factor that exceed expectations for level IV can receive a score as high as 5.9. In other words, contributions beyond what is expected for an employee at that level are more highly leveraged in terms ofscoring. The four factor scores are averaged to reach an employee's OCS. The Manual provides that [t]he maximum compensationfor broadband level IV is the GS15/Step-lOsalary and equates to a total OCS of justbelow 4.9. Ifthe average ofCCS factor scores exceeds 5.25, the total OCS

6Id. will be set to 5.25 with the individual identified to upper management as having exceeded the maximum contribution defined by the broadband. However, contribution exceedingthe level IV descriptors would not commonly occur.

After the First-Level MoM, the Pay Pool Manager ("PPM") takes part in what is referred to as the "PPM MoM." "The purpose ofthis meeting is

.

. . to ensure consistent application of the CCS process . . . ." Id. at KAPL000283. The PPM may direct subordinate supervisors to take a second look at scores ifhe or she finds thatthe CCS process has notbeen consistently applied. The PPM approves the scores after they have been finalized. Once the PPM has approved and finalized all assessments, the firstlevel supervisors are then required to conduct assessment feedbacks with the employee being evaluated. Id. The feedback session is expected to include "a discussion ofthe contribution statements, factor scores, OCS, developmental and career opportunities, and expectations for the upcoming cycle." Id. A copy ofthe "Annual Contribution Evaluation Form," known as "AFRL Form 280," is then given to the employee. The AFRL Form 280 "documents decisions made during the MoMto include factor write-ups and factor scores." Id. The grievance period begins after the employee receives his or her AFRL Form 280. Inthe absence ofa successful grievance, the employee's actual OCS is then compared to her "expected OCS," in order to determine whether she is being properly compensated. As mentioned above, the StandardPay Line can be used to generate an employee's expected OCS by determining the OCS score that correlates with the employee's currentbasic pay. Ifanemployee's actual OCS is greaterthantheirexpectedOCS, they are eligible for an increase in pay based on their increased contribution. The following tables demonstratethe expected OCS, actual OCS, delta OCS, and resulting pay for Dr. Kaplan and each ofher comparators for the relevant time periods. Given that Dr. Kaplan twice changedpositions several months into an assessment cycle, there is some overlap between the three tables. The salary amount indicated is for the calendar year immediately following a particular assessment cycle.

7Program Manager Assessment Expected Actual Delta Resulting Total CCS and Cycle 

## ocs ocs ocs Base Broadband IV Salary Bonuses Dr. Kaplan 10/01109- [ ] [ ] [ ] [ ] [ ]

9130110 10/01/10- [ 

### l [ l [ l [ 

### l [ l 9/30/11 Dr. Schlossberg

10/01109- [ 

### l [ 

### l [ 

### l [ 

### l [ ]

9130110 10/01/10- [ 

### l [ l [ ] 

### [ l [ l

9130111 Dr. Wu 10/01109- [ 

### l [ 

### l [ l [ ] 

### [ ]

9130110 10/01/10- NIA 9/30/11
PX 22, 29, 31, 32, 175; DX 100. Deputy Director Assessment Expected Actual Delta Resulting Total CCS and Cycle 

## ocs ocs ocs Base Broadband IV Salary Bonuses Dr. Kaplan I0/01/09- [ 

### l [ 

### l [ 

### l [ 

### l [ ]

9130110

810/01/10- [ ] [ ] 

### [ ] [ ] [ ] 9/30/11 10/01/11- [] [ ] 

### [ l [ ] [ ] 9/30/12 10/01/12- 

### [ l [] [ ] [ ] [ ] 9/30/13 Dr. Blackwood 10/01/09- 

### [ l [ ] [ ] [ ] 

### [ ] 9/30/10 10/01110- [ ] [ ] [ ] [ ] [ ] 9/30/11 10/01111- [ ] [ ] 

### [ ] [ ] [ ] 9/30/12 10/01/12- [ ] [ ] 

### [ ] [ ] [ ] 9/30/13 Dr. DeLong 10/01/09- [ ] [ ] [ ] [ ] [ ] 9/30/10 10/01/10- [ ] [ ] [ ] [ ] [ ] 9/30/11 10/01/11- [ ] [ ] [ ] [ ] [ ] 9/30/12 10/01/12- [ ] [ ] [ ] [ ] [ ] 9/30/13
Px 9, 10, 18, 19, 24, 25, 26, 27, 28, 31, 32; DX 100.

9Program Officer Assessment Expected Actual Delta Resulting Total CCS and Cycle 

## ocs ocs ocs Base Broadband IV Salary Bonuses Dr. Kaplan 10/01/11- [ ] [ ] [ ] [ ] [ ] 9130112 10/01/12- [ ] [ ] [ ] [ ] [ ] 9/30/13 10/01113- [ ] [ 

### l [ ] [ ] [ ] 9130114 10/01/14- [ ] [ ] 

### [ ] [ ] [ ] 9130115 Dr. Nachman 10/01/11- [ ] [ ] [ ] [ ] [ 

### l 9130112 10/01112- [ ] [ ] [ ] [ ] [ ] 9130113 10/01113- [ ] [ 

### l [ ] [ ] [ ] 9130114 10/01114- [ ] [ ] [ ] [ ] [ ] 9/30/15 Dr. Bonneau 10/01/11- [ ] [ 

### l [ ] [ ] [ ] 9/30/12 10/01112- [ ] [ ] [ ] [ ] [ ] 9130113 10/01113- NIA 9130114

10I0/01/14- 9/30/15 NIA

PX 9, 10, 20, 21, 36, 37, 39, 44, 45, 139; DX 100. Sixteenwitnesses testified during trial. We also receivedthetranscript ofthe de bene esse deposition ofColonel Robert Kraus, a former first-level supervisor of Dr. Kaplan. We will list the witnesses in order of their appearance along with a briefbiographical description. As relevant, we will discuss the particulars oftheir testimony in more detail below. On the first day of trial, four witnesses testified. The first was Dr. Kaplan. Second was Dr. Frederica Darema, former director of the Mathematics, Information, andLife Sciences Directorate and current director ofAFOSR. Third was Dr. Thomas Russell, former AFOSR director and the Pay Pool Manager ("PPM") for the assessment cycles that set Dr. Kaplan's 2011, 2012, and 2013 pay. Dr. PatrickCatTickwas the fourth witness to testify and was the PPMthat setDr. Kaplan'spay for 2014. Dr. Kaplan served as Dr. Carrick's deputy when he was the director of the Physics & Electronics Directorate. Onthe second day oftrial, Dr. Carrick concludedhistestimony and the following six witnesses testified. Ms. Lisa Aiken, a paralegal for the Department of Justice and the fifth witness to testify, helped prepare defendant's exhibit 100-agathering ofthe salaries and overall contribution scores ofplaintiffs and her comparators, along with a total oftheir bonuses, ifany. Colonel Andrew Szmerekovsky, the sixth witness to testify, was the PPMresponsible for the assessment cycle that set Dr. Kaplan's pay for 2015. Ms. Michelle Williams testified seventh. She was the chiefofthe personnel demonstration project office in the headquarters of the AFRL at all times relevant to this case. Eighth to testify was Ms. Terry Brailsford, who was at all relevant times the chief of operations management and responsible for human resources. Ms. Mary Jackson was the ninth witness. She was the contribution-based compensation system coordinator for the AFOSR Lab Demo System. Dr. Milton Blackwood, one ofplaintiffs alleged comparators for her deputy director position, testified tenth.

11On the third and final day oftrial, the remaining sixwitnesses testified: Dr. Hugh DeLong, one of plaintiffs alleged comparators for her deputy directorposition (he appeared by videoconference); Dr. RobertBonneau, one ofplaintiffs alleged comparators for her program officer position; Dr. Arje Nachman, one of plaintiffs alleged comparators for her program officer position; Mr. Raheem Lawal, not an alleged comparator but a program manager beginning in 2005; Dr. Thomas Christian, Jr., the PPM responsible for setting Dr. Kaplan's pay for 2016 and until December 2016, the director ofAFOSR; and Lieutenant Colonel Jamie Morrison, the branch chiefofthe Engineering and Information Sciences branch and Dr. Kaplan's first-level supervisorbeginning in April of2015. Thede bene esse deposition ofColonel Robert Kraus, Dr. Kaplan's first-level supervisor prior to Lt. Col. Morrison, was admitted as Court Exhibit 1. DISCUSSION The Equal Pay Act provides in relevant part that: No employer having employees subject to any provision ofthis section shall discriminate, within any establishment in which such employees are employed, between employees on the basis ofsex by paying wages to employees in such establishment at a rate less than the rate at which he pays wages to employees of the opposite sex in such establishment for equal work onjobs the performance of which requires equal skill, effort, and responsibility, and which are performed under similar working conditions, except where such payment is made pursuant to (i) a seniority system; (ii) a merit system; (iii) a system which measures earnings by quantity or quality ofproduction; or (iv) a differential based on any other factor other than sex .... 29 U.S.C. § 206(d)(l). Dr. Kaplan believes that, during her service as a program manager, deputy director, and program officer with AFOSR, the government violated this provision and did so willfully. Even though the limitations period, assuming a willful violation, goes back no further than January 2011, she can challenge the operation ofthe Lab Demo project as far back as October of2009, when the assessment cycle fixing her pay for 2011 began. To successfully prove a violation ofthe Equal Pay Act, she has the burden to demonstrate that she was paid less than her respective male comparators for ajobthatrequired substantiallyequal skill, responsibility, and
12effort. Cooke v. United States, 85 Fed. Cl. 325, 341 (2008). Ifplaintiffis able to establish herprimafacie case, the burden shifts to defendant to demonstrate by a preponderance of the evidence that the difference in pay is due either to "(i) a seniority system; (ii) a merit system; (iii) a system which measures earnings by quantity or quality ofproduction; or (iv) a differential based on any other factor than sex." 29 U.S.C. § 206(d)(l). The burden on the employer is "aheavy one," and it must demonstrate thatthe factor causing the difference in wage is actually the gender-neutral factor identified. Cooke, 85 Fed. Cl. at 347. The government attributesthepaydisparitybetweenDr. Kaplan andher male comparators to the proper functioning ofthe Lab Demo system, which was in place at all times relevant to this case, and which compensates employees according to the quality of their performance by measuring an employee's contribution to the AFOSR mission through an organized procedurethatuses predeterminedcriteria. Itcontends thatplaintiffwas fairly scored for her contribution under that system, and as a result of her lower contribution to the mission ofAFOSR, she continued to be paid less than her alleged comparators. To establish the merit system defense, defendantmust demonstratethat the Lab Demo "was an organized and structured procedure by which employees were evaluated systematically and in accordance with predetermined criteria. Under such a merit system, an employer may reward workers for outstanding experience, training, and ability, so long as the resulting salary differential is not basedupon sex." Raymondv. UnitedStates, 31 Fed. Cl. 514, 518 (1994) (internal citations omitted). Further, a meritbased system does not have to be devoid ofsubjectivity. EEOC v. AetnaIns. Co., 616 F.2d 719, 726 (4th Cir. 1980) ("An element of subjectivity is essentially inevitable in employment decisions; provided that there are demonstrable reasons for the decision, unrelated to sex, subjectivity is permissible."). Finally, "[i]fthe merit system is not in writing, the employer must also show that the employees were aware ofit." 7 Cooke, 85 Fed. Cl. at 347 (citingAetna, 616 F.2d at 725).

7 We note that the Lab Demo is in writing and that we are not persuaded by plaintiffs arguments that slight deviations or discretionary decisions by the supervisors amounted to a change to some unwritten policy. 13There is no question that plaintiff was paid less than the male employees she selected as comparators. Nevertheless, it is unnecessary for us to consider whether plaintiffhas established that these men performed work that required substantially equal skill, responsibility, or effort because the government has clearly established the second-listed defense above-the application ofamerit system. 8 We conclude for reasons set out below thatthe Lab Demo is a legitimate and comprehensive merit system that adequately explains the difference in pay between Dr. Kaplan and her alleged male comparators. The Lab Demo system is as free from subjectivity as one could reasonably expect a merit system to be. Also, we find that, in assessing both Dr. Kaplan and her alleged male comparators, her supervisors deviated from the Lab Demo procedures onlyto aminor and inconsequential degree, causing no prejudice to Dr. Kaplan. 9 As plaintiffpoints out, and as can be seen from the chart set out above, plaintiffs beginning salary in 2011, when she began her term as a Deputy Director, was approximately $9,000-$12,000 less than the other two deputy directors were making that year, not considering the differences in bonuses. Plaintiffargues that this illustrates her claim ofdisparate treatment. In response, the governmentpoints outthather salarycommencingthat yearwas the result ofthe assessment cycle for October2009-September 2010, during which time she was not yet a deputy director but was a program manager. The Lab Demo contemplates that in such a situation, her default

8 Not that we have any misgivings about the quality of the government's evidence on this point. There was ample evidence that plaintiff was not performing the work that required substantially the same skill, effort and responsibility. For example, her work as deputy director was more often administrative in nature, whereas Dr. DeLong served as acting director ofhis directorate at this time, while also managing aportfolio. Dr. Bonneau, one of her comparators for the program officer position, in addition to his other duties, served as "cochair ofthe large-scale networking group for the Office ofScience and Technology Policy for the White House." Tr. 737.

9 For example, Dr. Kaplan testified that she only received mid-cycle feedback for two assessment cycles relevant to this litigation. Tr. 78-79. Because this mid-cycle review was not a part ofthe end-of-cycle assessment process that determined her pay, this deviation from the Lab Demo carries little weight in plaintiffs attempt to rebut the government's affirmative defense. 14salary would be carried along from the prior position, as adjusted by the assessment results. Although supervisors had the discretion, when someone moved positions in that fashion, to raise the base pay in anticipation of a greater expected contribution, that did not occur in her case. Despite her change in title, her salary as Deputy Directorthus was determined by herprior performance in the program manager position. Thishappened again in February2013 whenherposition changed from deputy directortoprogram officer. Herpay as aprogram officer commencing in that year was fixed by her evaluation for the preceding assessment cycle when she was working as a deputy director. There is no reason to think that plaintiff was treated any differently in this respect than anyone else in the agency who moved between positions. All ofthis was consistent with the manual. 10 The mere fact that she held the same title as her higher paid comparators is not an indictment of the system. The relevant inquiry is whetherthe agency's application ofthe assessment process was legitimate. If itwas, thenthe disparity in salary from yearto yearwould continue. Giventhe nature ofthe Lab Demo, it is inevitable that employees with the samejob title will receive different salaries basedontheirrespective contributions and those differences are cumulative. We also note thatthe Lab Demo is more agilethan the GS systemwhen it comes to recognizing an employee's impactful work with higher compensation. For example, Dr. Bonneau, an alleged compartator for Dr. Kaplan's program officer position, joined the AFOSR as a DR-III level program manager in 2007. Tr. 730. While serving as aprogram manager, Dr. Bonneau quickly raised his OCS score to the [ ] maximum in the assessment cycle beginning October 1, 2010. PX 43. This represented a nearly [ ] OCS

10 The manual gives management the authority to make such a change in pay when an employee moves within the system but does not make it mandatory. PX 50 at KAPL000272. The Manual provides that "[w]hen settingbasicpay

... managers must consider an employee's expected level of contribution based upon academic qualifications, competencies, experience

... [and their] current OCS" among other factors. Id. We decline the opportunity to second guess her supervisors as plaintiffhas failed to show how their decision was irrational or otherwise inconsistent with the Manual. 15increase from his expected OCS of [ ] from the previous cycle. PX 42. In other words, in two years, Dr. Bonneau was able to raise his Actual OCS by a significantly greater amount than Dr. Kaplan did through all assessment cycles relevant to this litigation. 11 The actual fairness and consistency of this somewhat mechanical grading scheme is thus dependent in the first instance on the quality ofthe first-line supervisors' scoring and then on whether there was meaningful second-level review. Here we have no reason to question either the inputs-admittedly somewhat subjective-or the higher-level reviews. Through testimony elicited at trial and other evidence in the record, the government was able to demonstrate that each of Dr. Kaplan's first-level supervisors, namely, Dr. Carrick, Colonel Kraus, and Lieutenant Colonel Morrison, used the required rubric and did not consider Dr. Kaplan's gender while preparing their assessments. Court Ex. 1 at 11-12, 24-26; Tr. 381, 385, 890, 903. For example, Lieutenant Colonel Morrison testified about his use ofthe Lab Demo rubrics during his assessments as follows: "what I'm doing is I've got that rubric in front ofme and I'm trying to match what language [from the rubric] best describes the activity that I've seen." Tr. 890. These first-line evaluations, which arguably could be the most problematic in terms of subjectivity, were then reviewed by second-line supervisors, the PPMs. Dr. Russell, Dr. Carrick, Colonel Szmerekovsky, and Dr. ChristiantestifiedthatbothDr. Kaplan's andhercomparators' OCS scores accurately reflected their contribution, and were in line with those of other employees. Tr. 258-59, 409-12, 492, 830-31. In addition,AFOSRadministrators,Ms. TerryBrailsfordand Ms. Mary Jackson, explained that they observed the meetings ofmanagers in order to ensure a level ofuniform application ofthe procedures. Ms. Brailsford, the ChiefofOperations Managementwho attended all MoMs since at least 2010, testified that the standardized rubrics were used in the MoMs to assess each

11 While the system allows for more rapid progression, it also contemplates situations where an employee is unable to further increase her contribution. The Manual provides that "[i]t is reasonable to expect an employee's ability to increase contribution to eventually plateau

.... In [this] case[], artificial bumps that increase scores (such as from a 2.2 to 2.3) should be avoided if contribution remains the same." Id. at KAPL000283. 16employee's contributions. Tr. 627. Ms. Jackson, the CCS coordinator who also attended all MoMs at all times relevant to this litigation, testified, as did Ms. Brailsford, that gender did not play a role in the Lab Demo process. Tr. 627, 655. Plaintiff cannot point to any credible evidence to rebut this testimony. In effect, her challenge amounts to no more than an effort to have the court substitute its assessment of her performance for that of her supervisors. Dr. Kaplan, in herpost-trialreply brief, identifies anumberofinstances in which she believes that the government's reliance on the PPMs' testimony is misplaced due to alleged inconsistences in their statements at trial. We are not persuaded, however. As an example, plaintiff argues that "Col Szmerekovsky did not address the 'red flags' with Col Kraus using copiedLab Demo instruction text as Dr. Kaplan's contributions." Pl.'s Post-Trial Reply 14. 12 However, it was only Dr. Kaplan who identified the copying itselfto be a "red flag," not Colonel Szmerekovsky. The only "red flag" that Colonel Szmerekovsky clearly identified inhis testimony was that Colonel Kraus was signaling that Dr. Kaplan's contribution level in certain areas was below the level at which Dr. Kaplan was expected to contribute as a DR-IV employee. Tr. 469-70. Dr. Kaplan makes a number ofarguments attemptingto show that the Lab Demo procedures were not followed. In all ofthem that are actually relevant to this case, plaintiffis either mistaken in fact or plaintiff's comment highlights an inconsequential deviation thatwe find was notprejudicial to her participation in the Lab Demo process. We hold that plaintiffhas not shown that the agency failed to adequately follow the procedures ofthe Lab Demo. Plaintiff also advances several arguments that we find amount to nothing more than her disagreements with how her contributions were valued

12 This is areference to Colonel Kraus's replacement ofsome ofthe language from Dr. Kaplan's self-assessment with that ofthe rubric's description ofa lower broadband level during the end-of-cycle assessment that set herpay for 2015. Dr. Kaplan also argues that, since Colonel Kraus copied the language, his assessment was not based onher actual contribution. We are satisfiedthat this was Colonel Kraus's way ofdescribing her actual contribution and note that her AFRL Form 280 from that year also mentions a number ofspecific contributions that she made. See PX 37. Moreover, as defendant points out, Ms. Michelle Williams testified that she did not find Colonel Kraus's taking excerpts from the rubric inthe manner thathe didto be inappropriate. Tr. 588. 17through the Lab Demo process. She fails to demonstrate, however, that there was an impermissible level ofsubjectivity. Forexample, in herpost-trialbrief, Dr. Kaplan takes issue with how Dr. Christian only pointed out "one thing" when he "was asked to compare Dr. Kaplan's impressive communications contributions which only received a [ ] with Dr. Nachman's less-thanimpressive [communication] contributions which received a [ ] ." Pl.'s PostTrialBr. 38. The one contribution thatDr. Kaplan is referencingrelated to Dr. Nachman's visit to the Intelligence Advanced Research Projects Activity ("IARPA"). Dr. Christian testified that this communication was particularly important because IARPA exists to support the intelligence community, "so it's a very important organization doing work that highly benefits the nation." Tr. 841. Dr. Christian went on to explain that "[t]here is tremendous money that IARPA puts into research, and this allows the AirForce to let them know whatwe're doing and inform them." Id. Dr. Kaplan cites this to illustrate the fact that one contribution can carry a lot ofweight while another may have little influence on a factor score, thus showing the "complete subjectivity" of the system. Pl.'s Post-Trial Br. 38. We disagree with her characterization. Indeed, it is difficult to imagine any merit-based system passing musterunder Dr. Kaplan's view. With respect to her position as a program officer, we note that the assessmentsperformedby Dr. Kaplan's supervisorswerecorroborated, inpart, by unfavorable reviews by the Air Force Scientific Advisory Board ("SAB") in front of which Dr. Kaplan presented in March 2016. 13 Dr. Christian described the SAB as follows: "The SAB is a very ... independent, objective, and I'11 use the term 'eminent experts.' They're from universities. They're from government laboratories

.... And they review every directorate within the research laboratory, of which AFOSR is one." Tr. 846-47. When Dr. Kaplan assumed the program officerposition in February 2013, she inherited the "Software and Systems" portfolio from Dr. Bonneau. Tr. 740. Relevant here, the SAB reviewedthe Software and Systems portfolio when itwas under Dr. Bonneau in 2012 and under Dr. Kaplan in 2016. In 2012, Dr. Bonneau received the highest possible rating from the SAB for the Systems and

13 We note that a presentation in March of2016 would be relevant for setting Dr. Kaplan'spay for2017, and thatDr. Kaplanwas removed from herposition in November of2016. We find the SAB 's assessment to be relevant in that it demonstrates the negative impact that Dr. Kaplan had over time on an inherited program. 18Software program. PX 44. Under Dr. Kaplan in 2016, the SAB found that it "was the only one to behighly deficient," and "strongly recommended that no money beputinto it until itwas restructured." Tr. 848. Two months later, Dr. Kaplan was placed on a contributionimprovementplanbased ona finding that her contributions were not commensurate with her pay. PX 213. While wehave discussed only a few examples ofthe evidenceplaintiff eites to demonstrate bias, we believethat they are illustrative and are satisfied that the difference in the contribution scores are not the result ofan improper amount of subjectivity. The Lab Demo system is structured to limit subjectivity,whilerecognizingthatthe assessments offirst-line supervisorsare to some extent subjective. The First-Level MoM, PPM MoM, and the employees' ability to later file grievances all work towards introducing objective points ofcomparison. As Dr. Christian testified, the "seeond set of eyes" provided by the other supervisors in the First-Level MoM minimize subjectivity. Tr. 835-36. We are also satisfiedthatthe deseription by Colonel Szmerekovsky, the PPM that set Dr. Kaplan's pay for 2015, is accurate. He viewed it as hisjob in the PPM MoMto "ensure the integrity ofthe system,"Tr. 468, and to make certain that there was a similarorconsistentstandardwhen [the supervisors are] looking at the rubric across the board

....And ifit's not consistent, [if] one person appears to be too hard on their people and one person appears to be too easy, we try to normalize it

...through discussion and say, hey, why are you rating this because

...this doesn'tmatch the rubric or this isn't consistent with howwe've been providing ratings. Tr. 519. Finally, plaintiffalso challengeswhat she considersto be inconsistency in the award ofbonuses. For example, Dr. Kaplan argues thatDr. Russell, the PPMforthe assessment cycles that setDr. Kaplan's 2011, 2012, and 2013 pay, could not explain how Dr. Nachman and Dr. Bonneau, who both received an Overall Contribution Score of [ ], were awarded different bonuses. However, as the government points out, Dr. Russell was able in a brief examination of Dr. Bonneau's AFRL Form 280 to identify at least one illustration that Dr. Bonneau was working beyond his portfolio and
19contributing at levels higher than Dr. Nachman, thus entitling him to a larger bonus. Tr. 212. In any event, while there was some uncertainty regarding the bonuses expressed during trial, we are satisfied that thetwo types ofbonuses were adequately described in the Manual, which was consistent with Ms

. Williams' testimony, among others. 

### See e.g., PX 50 at KAPL000272-73, 000310; Tr. 579

. We also notethatDr. Kaplan'sactual OCS nevercameclose tothe4.85 score thatwouldmakethe broadbandIVbonus relevantto hertotal compensation. Also, she has failed to demonstrate any unsustainable contribution for which she should have been awarded a CCS bonus. Accordingly, we see no reason to question her supervisor'sjudgment in not awarding her a bonus. CONCLUSION Dr. Kaplanwaspaidless thanheralleged comparators,butevenifthose men did work performed under similar working conditions that required substantiallyequal skill, effortand responsibility, thegovernmenthas satisfied it burden ofproving that those pay differences are entirely attributable to the operation ofits merit pay system. We have considered all of Dr

. Kaplan's other arguments and find them to be without merit. 14 Accordingly, plaintiffs complaint is dismissed. The Clerk ofCourt is directed to enterjudgment in favor ofthe United States

. No costs.

# ~ Senior Judge

14 Plaintiffargues that shewas denied a number ofadvancement opportunities that she believes would have helped her bolster her contribution. We are satisfied that her supervisor's actions in this regard were based on legitimate management considerations and are not a reflection of gender bias. In her post-trial brief,plaintiffalso makes a general argumentregarding otherfemale employees thatwerepaid less than the male employees they replaced

.Attrial, Dr

.Kaplan made a relevance objectionwhen defendant's counselbegana line of questioning designed to elicit facts that could rebut this argument. We sustained plaintiffs objection after she stated that she would not bepursuing such an argument. Tr. 291. 20