Opinion ID: 501743
Heading Depth: 1
Heading Rank: 2

Heading: disparate treatment--hiring and promotions

Text: 17 The EEOC argues that the district court erred in determining that it did not show a pattern or practice of discrimination by Sears against women in hiring and promotions into commission sales positions. 9 We base our decision on the well-reasoned and comprehensive opinion of the district court, see Sears II, 628 F.Supp. at 1277, and address the principal and dispositive issues raised by the parties. Although in Coates v. Johnson & Johnson, 756 F.2d 524, 533 (7th Cir.1985), this court briefly reviewed all the evidence in that disparate treatment case, it is not expedient or necessary to do so here, considering the variety and amount of statistical and nonstatistical evidence and the detailed summary of all the evidence by the district court in its opinion on the merits. Although we consider all the evidence, we will review the evidence here only to the extent necessary to consider the EEOC's contentions that the district court has clearly erred in reaching various factual findings. 18 The nature of the evidence presented by each party in this case was significantly different. It is helpful to briefly overview the types of evidence presented by the parties. The EEOC presented, almost exclusively, statistical evidence in the form of regression analyses based on information from employment applications of rejected sales applicants and Sears' computerized payroll records from 1973 through 1980. The EEOC based other regression analyses on information from Applicant Interview Guides Sears had administered at various times from 1978 through 1980 at two Sears stores in its Southwestern Territory. The EEOC attempted to bolster this statistical evidence through nonstatistical evidence regarding the subjective nature of Sears' selection process and allegedly discriminatory aspects of Sears' testing practices. 19 Sears did not respond with like regression analyses based on employment application and payroll records. Instead, most of Sears' evidence was directed at undermining two assumptions Sears claimed were faulty and fatal to the validity of the EEOC's statistical analysis--the assumptions of equal interests and qualifications of applicants for commission sales positions. This evidence consisted of testimony by Sears store managers, personnel managers, and other store officials, a study based on interviews of women in nontraditional jobs at Sears, national surveys and polls regarding the changing status of women in American society, morale surveys of Sears employees and 1976 and 1982 job interest surveys of Sears employees, national labor force data, and an analysis of the Applicant Interview Guides that attempts to measure differences in interest among men and women. Sears also presented evidence regarding its hiring figures, general evidence regarding the characteristics of commission salespersons including a case study of all commission sales hires in all stores, based on information in personnel files of applicants who were hired and sales performance data, evidence regarding the employment milieu at Sears, especially relating to commission selling and the structure of Sears, and evidence regarding its affirmative action efforts. 20 Regarding the claim of discrimination in promotions, the EEOC again relied almost totally on regression analyses based on female proportions of noncommission salespersons in each store at the end of a year compared with the proportion of women actually hired into commission sales. Sears again relied on the evidence relating to interest, the nature of commission sales, characteristics of commission salespeople, and its affirmative action efforts. In addition, Sears introduced further evidence of interest in the form of responses to Career Aspirations Questionnaires it had administered to noncommission salespersons, and performed various adjustments of the EEOC's statistical analysis. 21 Each side featured one primary expert on statistical evidence--the EEOC starred Dr. Bernard R. Siskin in this role, while Sears was represented by Dr. Joan G. Haworth. 22 The EEOC argues that Sears' generalized interest evidence is inadequate as a matter of law to refute the EEOC's statistical presentation. We have already partially discussed this argument in the context of the burdens of proof, but there is another aspect of this argument. The EEOC implies that Sears had the burden of responding with a more probative statistical analysis. The Supreme Court in Teamsters specifically stated, however, that [w]e do not ... suggest that there are any particular limits on the type of evidence an employer may use. 431 U.S. at 360 n. 46, 97 S.Ct. at 1867 n. 46. An employer may attempt to show that plaintiffs' proof is either inaccurate or insignificant, id. at 360, 97 S.Ct. at 1867, or the plaintiff may attempt to provide a nondiscriminatory explanation for the apparently discriminatory result. Id. at 361 n. 46, 97 S.Ct. at 1867 n. 46. Then-Justice, now Chief Justice Rehnquist, concurring in Dothard v. Rawlinson, 433 U.S. 321, 338-39, 97 S.Ct. 2720, 2731, 53 L.Ed.2d 786 (1977), stated that defendants in a discrimination case may endeavor [in rebuttal] to impeach the reliability of the statistical evidence, they may offer rebutting evidence, or they may disparage in arguments or in briefs the probative weight which the plaintiffs' evidence should be accorded. See also Catlett v. Missouri Highway & Transportation Commission, 828 F.2d 1260, 1266 (8th Cir.1987) (defendant may introduce evidence that a lesser interest in certain jobs on part of female applicants explains statistical disparity); Segar v. Smith, 738 F.2d 1249, 1268 (D.C.Cir.1984) (defense [t]ypically ... will focus on the integrity of the plaintiffs' statistical methodology and the significance of the results shown), cert. denied, 471 U.S. 1115, 105 S.Ct. 2357, 86 L.Ed.2d 258 (1985). The defendants' rebuttal must ... at least raise a genuine issue of material fact concerning the accuracy of the picture painted by the plaintiffs' statistics. Segar, 738 F.2d at 1268. The cases cited by the EEOC to support its argument that Sears had the burden of rebutting its statistical analysis with more refined, accurate and valid statistical evidence did not state that the defendant must produce such evidence to succeed in rebutting the plaintiffs' case. Instead, those cases indicated that a defendant could or was entitled to use such a means of rebuttal. See Shidaker v. Carlin, 782 F.2d 746, 750 (7th Cir.1986), vacated, --- U.S. ----, 107 S.Ct. 1621, 95 L.Ed.2d 195 (1987); Movement for Opportunity & Equality v. General Motors Corp., 622 F.2d 1235, 1245 (7th Cir.1980). These cases involve disparate impact claims--without deciding whether these principles are applicable in a disparate treatment case, we can say that EEOC misconstrues the principles. These cases suggest, and the cases we have cited above confirm, that statistical evidence is only one method of rebutting a statistical case. Cf. Teamsters, 431 U.S. at 340, 97 S.Ct. at 1827 (Statistics are not irrefutable; they come in infinite variety and, like any other kind of evidence, they may be rebutted. In short, their usefulness depends on all of the surrounding facts and circumstances.). We therefore reject the EEOC's contention that Sears' interest evidence, consisting of testimony of Sears' store witnesses, external labor force data, national survey data, and data from surveys of Sears' employees, is insufficient as a matter of law to undermine the EEOC's statistical evidence. 23 Before we discuss the specifics of the EEOC's challenges to the district court's findings regarding the EEOC's hiring and promotion claims, we address the EEOC's arguments that the district court erred as a matter of law in analyzing those claims. The EEOC contends that the district court erred as a matter of law by failing to address separately a period of liability the EEOC terms the early years. Second, the EEOC argues that the court erred by not considering its claims of discrimination regarding part-time hires separately from its claims of discrimination regarding full-time hires.
24 The EEOC argues that the district court erred as a matter of law by failing to address separately what the EEOC terms the earlier years at issue in its hiring and promotion into commission sales claims. In support of this argument, the EEOC cites various cases it says stand for the proposition that [t]he law clearly requires that a defendant's employment practices prior to its taking corrective action be examined separately. We do not read the cases as supporting that particular proposition. Instead, we believe the cases hold, at most, that an employer's corrective action does not absolve the employer from liability based on discriminatory practices occurring before the employer took the corrective action. See Teamsters, 431 U.S. at 341-42, 97 S.Ct. at 1857-58; Rich v. Martin Marietta Corp., 522 F.2d 333, 346 (10th Cir.1975); Parham v. Southwestern Bell Telephone Co., 433 F.2d 421, 425-26 (8th Cir.1970); Patterson v. Youngstown Sheet & Tube Co., 440 F.Supp. 409, 412-13 (N.D.Ind.1977) (defendant employer not permitted to combine statistics for precharge and post-charge periods, because [a]ctions taken subsequent to the filing of E.E.O.C. charges cannot moot issues of discrimination occurring prior to that time), aff'd, 659 F.2d 736 (7th Cir.), cert. denied, 454 U.S. 1100, 102 S.Ct. 674, 70 L.Ed.2d 641 (1981); cf. Capaci v. Katz & Besthoff, Inc., 711 F.2d 647, 655-57 (5th Cir.1983), cert. denied, 466 U.S. 927, 104 S.Ct. 1709, 80 L.Ed.2d 182 (1984). These cases are all characterized by an assumption that employer practices in existence before some form of corrective action (usually occurring after Title VII charges were filed against the employer) were in fact discriminatory. 25 The district court in this case, however, never found that Sears engaged in discriminatory hiring and promotion practices before the EEOC filed charges or before Sears began affirmative action efforts. The district court also never held that Sears' later policies and programs absolved Sears of any liability for earlier discriminatory practices. Instead, the district court found that the EEOC's statistics failed to prove liability for any period at issue in this case and that Sears' affirmative action efforts began as early as 1968, well before the EEOC's August 1973 charge. 26 The EEOC's challenge is thus not that the district court erred as a matter of law. Its argument is more properly characterized as a claim that the court incorrectly failed to make two factual findings, either finding of which should have led the district court to consider the early years separately from the later period. One of those findings the EEOC claims the district court erred in not making was that there were significant differences between the early years and the later years in the liability period of 1973 to 1980. The second finding would have been that Sears adopted the Mandatory Achievement of Goals (MAG) affirmative action plan in response to the EEOC's charge, or that Sears designed the MAG program as a corrective action to bless earlier discriminatory practices. On review of factual findings, we will uphold the district court's findings unless they are clearly erroneous. See Anderson v. City of Bessemer City, 470 U.S. 564, 573, 105 S.Ct. 1504, 1511, 84 L.Ed.2d 518 (1985); Andre v. Bendix Corp., 774 F.2d 786, 788 (7th Cir.1985). We are not persuaded that the district court's failure to make either of the above findings was clear error. 27 The EEOC claims that during the early years there were fundamental changes in Sears' practices regarding the hiring and promotion of women into commission sales positions. The EEOC then cites statistics indicating that in various sales departments (e.g., auto accessories or furniture), percentages of commission saleswomen doubled, tripled, quadrupled, or increased up to eightfold during various early periods. Initially, we note that the district court found that all of the EEOC statistics were plagued by arbitrary and false assumptions. We discuss these statistical defects in more detail later. Because we find that the district court did not err in its criticisms of the EEOC's statistics, we cannot accept the EEOC's argument that those statistics indicated differences that the district court should have recognized by separating early years from later years. 28 Even if the district court had found the EEOC's underlying statistics sound, we are not convinced by the EEOC's arguments that the district court should have recognized differences between early years and some later period. First, we are unsure what the EEOC means by the term early years, because the EEOC never specifically defined that term. The district court designated the liability period as beginning March 3, 1973 and ending in 1980. At various points in its argument, the EEOC suggests that the early years is a period from 1972 (the year before the EEOC charge was filed) to 1975 (two years after the charge was filed), the period before August 30, 1973 (when the EEOC filed charges), a period from 1973 to 1975, or the period before the summer of 1974 (when Sears implemented the MAG program). 10 Because the EEOC ultimately wants to argue that it can show that Sears discriminated before the implementation of its affirmative action program by separating the figures before and after implementation of affirmative action, we believe the EEOC means the early years as the beginning of the liability period, which the district court found was March 30, 1973, to the implementation of the MAG program in the summer of 1974. 11 Assuming that this fifteento sixteen-month period constitutes the early years, we are not convinced by the EEOC's arguments and statistics that the district court erred by failing to recognize significant differences between this period and the years following it. 29 The EEOC did not emphasize this period in its complaint, nor did it present data corresponding to this fifteen- to sixteen-month period that it now claims should have been considered separately. Neither did it attempt to distinguish data for promotions and hires during this period from data for promotions and hires during the time beginning after the early years ended. Instead, the statistics it cites to us on appeal (which were also presented to the district court) as indicating dramatic changes in hiring and promotion encompass periods before and after the early years. The EEOC presented data on promotions and hires broken down by year, beginning with 1972. 12 Its figures for 1973 thus included two months of hires and promotions before the relevant liability period began, and its data for 1974 included five to six months of hires and promotions that occurred after the MAG plan was implemented. 30 The data compiled and presented to this court by the EEOC are misleading as well as imprecise. The EEOC, in citing statistics it says show dramatic increases during the early years, does not carefully recognize or refer to that time period. For example, it states that between 1972, the year before the EEOC charge was filed, and 1975, two years after the charge was filed, the female proportion of full-time commission sales hires more than tripled (from 9.9% to 31.1%) and in breaking down the figures by division, that in Sewing Machines, during this three-year period, the female proportion of full-time hires almost tripled (from 13.5% to 47.9%) and in Furniture, it more than quadrupled between 1972 and 1974 alone (8.1% to 33.8%) and had increased more than fivefold (to 43.0%) by 1976. These statistics obfuscate increases that took place during the early years, before the implementation of the MAG plan. Although the EEOC claimed that the female percentage of sales hires more than tripled between 1972 and 1975, the same percentage nearly doubled between 1972 and 1973. Analysis by division of products is similarly misleading--percentages of Sewing Machines hires that the EEOC says almost tripled between 1972 and 1975 more than doubled between 1972 and 1973 and percentages for the Furniture division that more than quadrupled between 1972 and 1974 tripled between 1972 and 1973. 13 31 Some of the statistics the EEOC cites are for a time period that approximates the early years, but fail to support its argument that there were differences before and after implementation of the MAG plan. The EEOC claimed that between 1973 and 1974 alone, which would be during the early years period, the female percentage of hires more than tripled in Hardware and increased almost twenty times in Plumbing and Heating (the latter figure almost trebling by the next year). Other EEOC compilations of statistics are misleading for different reasons. The EEOC claims that the female percentage of commission sales hires for part-time outside the Midwestern Territory more than doubled between 1972 and 1975. A close look at the data indicates that the percentage was increasing steadily from 1972 to 1974, including during 1973-1974, which is fairly close to the early years period at issue. While some of the data seems to support the differences claimed by the EEOC--for example, the female percentage in Auto Accessories increased more than eightfold between 1972 and 1974 and more than twelvefold between 1972 and 1975--the difference in percentage increases between 1972-1974 and 1974-1975 is not as great as the EEOC's compilation of data suggests. In short, the EEOC's attempts to show significant differences between pre-MAG and post-MAG percentages of female workers are insufficient to persuade us that the district court clearly erred in failing to recognize the alleged increases by considering the early years separately. 14 32 The second aspect of the EEOC's claim that the district court should have separately considered the early years is that the district court clearly erred in failing to find that Sears' implementation of the MAG plan was corrective action taken by Sears in response to the EEOC's charges to escape liability for its earlier discrimination. The district court found, however, that Sears' commitment to affirmative action began in 1968, Sears in 1969 set a long-term goal of 38% women at all jobs at Sears, and in April 1970, Sears instituted a centralized, company-wide affirmative action program for women and minorities, which was revised in 1972. Sears II, 628 F.Supp. at 1293. The EEOC argues that nothing in the record supports the district court's findings that attracting women to commission sales at Sears has been an important priority in Sears' affirmative action program since the first affirmative action questionnaire was circulated in 1968 and that Sears had engaged in pre-MAG efforts to hire or promote women into commission sales. Yet the EEOC acknowledges that in 1972 Sears' affirmative action plan mentioned the movement of women into commission sales positions. The year 1972 is pre-MAG, because, as the court found, MAG was implemented in mid-1974. The EEOC also fails to refute the district court's finding that Sears set a long-term goal of 38% women at all jobs at Sears in 1969--presumably all jobs includes commission sales jobs. 33 The EEOC attempts to characterize the MAG plan as a response to its charge against Sears by quoting the remarks of Sears Chairman Arthur M. Wood at a meeting of top executives to discuss affirmative action. Wood stated: 34 The fact is that Sears has been identified as a target by the [Equal Employment Opportunity] Commission and we must improve our situation quickly in order to be in as defensible position as possible--to do this I want to urge our lawyers and others working on discrimination cases to conciliate every possible claim; in short, we should stay out of court and out of direct confrontation with enforcement agencies as long as possible, giving our program of voluntary action time to show meaningful progress for minorities and women. 35 This remark, however, is not proof that Sears implemented MAG in response to the EEOC's charge. The remark was made May 21, 1973, more than three months before the charge was filed. On its face the remark indicates nothing more than that Sears was aware of potential EEOC action, although the EEOC claims that Wood had been informed by Ray J. Graham, Sears' Director of Equal Opportunity from 1968 through 1980, of the likelihood of the EEOC's bringing a National Programs charge against Sears. Other evidence is consistent with the court's finding that Sears' affirmative action efforts began in 1968. Graham testified, I can't emphasize too strongly that it was not the threat of EEOC action that caused Sears to [adopt the MAG plan], and stated that at the time of the meeting he had no knowledge of any EEOC targeting of Sears. Graham further testified that the MAG program was designed to accelerate [Sears'] progress and continue Sears' leadership role in affirmative action, and that Sears had one [affirmative action] plan from the very outset [in 1968]. Furthermore, the EEOC never identified at trial the policies or practices of the MAG plan that allegedly changed the disparities generated by its statistical analyses. Apart from saying that MAG was materially different from earlier affirmative action programs, the EEOC does not state what the changed practices were. The EEOC has not persuaded us that the district court clearly erred in finding that Sears' affirmative action commitment began in 1968. We hold that the court did not err in not considering the early years separately.
36 The EEOC also argues that the district court erred as a matter of law in failing to consider the EEOC's claims of discrimination regarding part-time hires separately from its claims regarding full-time hires. The EEOC cites no authority for this proposition, but argues that separate consideration of part-time from full-time is compelled by significant differences between part time and full time hires. This is not a question of law; rather, the EEOC must be arguing that the district court clearly erred in failing to find the alleged fact that differences between part-time and full-time hires were significant enough to warrant separate discussion of the part-time figures. One difference, according to the EEOC, is that the mix of products sold was different between full- and part-time, which the EEOC claims is significant because the court based much of its opinion on the nature of the products sold. The other difference mentioned by the EEOC is that Siskin's multicell analysis produced larger disparities for part-time hires than for full-time hires. 37 The district court did not, as the EEOC appears to suggest, ignore or never consider part-time figures. Neither did it completely subsume the part-time within the full-time case. At various points in its opinion the court referred to part-time data and disparities as distinct from full-time data and disparities. It appears that the court did refer to the full-time case more than it did the part-time case, but that may be explained by the fact that the full-time case was emphasized more at trial. 15 Indeed, the court explicitly stated that [f]ull time and part time positions ... were ... analyzed separately [by the parties]. The court will structure its analysis accordingly. Sears II, 628 F.Supp. at 1288. 38 Although the EEOC analyzed part-time figures separately from full-time figures, it employed the same mode of analysis to part-time and full-time data. Thus the part-time statistics were subject to the same criticisms by the district court as were the full-time statistics. We address these flaws recognized by the district court in more detail later. At this point it is sufficient to note that the part-time analyses, like the full-time analyses, are fraught with design flaws as well as a failure to capture differences in male and female interests and qualifications. In addition, Sears' defense of affirmative action applied to both full- and part-time analyses. To the extent that the part-time figures as well as the full-time figures are misleading and imprecise, the EEOC's claim that differences between part-time and full-time disparities were significant enough to warrant separate consideration of the part-time disparities is without merit. 39 The EEOC also argues that the district court impermissibly failed to recognize a distinction between part-time and full-time hires based on differences in product lines. The EEOC criticizes two of the district court's statements that give the impression, according to the EEOC, that the district court believed commission selling involved solely big ticket items such as major appliances. Such an impression would lead to false conclusions according to the EEOC, because commission selling involved smaller items like shoes as well as big ticket items like furnaces, and big ticket product lines were a small part of part-time hires compared with smaller product lines like shoes. 40 We are unconvinced that the district court clearly erred in failing to recognize a distinction between part-time and full-time figures based on certain product lines in commission sales. First, we do not believe as the EEOC claims, the court based much of its opinion on the nature of the products sold. The two statements cited by the EEOC in support of its claim are [c]ommission selling usually involved 'big ticket' items, meaning high-cost merchandise, such as major appliances, furnaces, air conditioners, roofing, tires, sewing machines, etc., Sears II, 628 F.Supp. at 1289, and the product lines into which 95 percent of Sears' full time commission salespersons were hired included such items as hardware, building supplies, paint and appliances. Id. at 1313 n. 58. These statements do not exclude product lines in which the EEOC claims there was a more persuasive part-time case. Second, these statements are insufficient to convince us that the nature of the products sold was crucial to the district court's reasoning. These are two statements in a seventy-six-page published opinion. We do not believe that the district court failed to address the EEOC's claims of discrimination against part-time employees.
41 The district judge found a plethora of problems in the statistical analyses that the EEOC had offered to support the claim that Sears discriminated against women in hiring into commission sales positions from 1973 to 1980. Before addressing the EEOC's specific challenges to the district court's criticisms of its statistical evidence, it is helpful to discuss three key findings made by the district court, which we believe are not clearly erroneous. Those findings are that during the period at issue in this case (1973-1980): (1) commission selling was significantly different from noncommission selling at Sears; (2) women were not as interested in commission selling as were men; and (3) women were not as qualified for commission selling as were men. 42 The finding that colors the district court's entire treatment of the EEOC's hiring as well as its promotion claims is that selling on commission at Sears is a very different job from regular, or noncommission selling at Sears. We cannot say that finding is clearly erroneous. The court's description of commission and noncommission selling at Sears indicates that the two forms of selling differed in the type of merchandise sold, the risk involved, which was reflected in the manner of compensation, and the technical knowledge, expertise, and motivation involved. The district court describes the differences at length, see Sears II, 628 F.Supp. at 1289-90, thus we need only mention major differences. As the district court found, commission selling at Sears usually involved selling big ticket items, which are high-cost merchandise such as major appliances, furnaces, roofing, and sewing machines. Merchandise sold on a noncommission basis understandably was generally low-cost and included apparel, paint, and cosmetics. Commission selling involved some risk, especially before 1977. During that period commission salespersons generally received a commission ranging from 6% to 9% percent plus a draw each week. The draw usually did not exceed 70% of average or estimated earnings, but was subject to reduction if the employee's commission did not equal the amount of the draw. There was always a risk that the employee could lose some of the draw if the commissions did not equal the amount of the draw. 16 After 1977, commission salespersons no longer faced deficits. In what the court noted was an effort to reduce the financial risk of selling on commission in an effort to attract more women to commission sales, Sears paid commission salespersons a nominal salary plus a 3% commission. Id. at 1289. Noncommission salespersons were paid on a straight hourly rate, and full-time salespersons received 1% commission on all sales until January 1979 when the practice was discontinued. The district court found that commission selling often required salespersons to be available after the normal working hours of 8:00 a.m. to 5:00 p.m., sometimes required that they sell in people's homes, might require a license depending on the products sold, and required qualities usually not as necessary in regular selling, including a high degree of technical knowledge, expertise, and motivation. 43 The court's next two major findings, that there were different interests and qualifications among men and women for commission selling, were grounded in part on the court's recognition of differences between noncommission and commission selling at Sears. The court based these findings on the large amount of evidence presented by Sears on these issues. The court extensively discusses this evidence. Id. at 1305-15. Again, we cannot say that these findings are clearly erroneous. 44 Regarding the question of differing interests in general among men and women in commission selling, we have already briefly reviewed the types of evidence presented by Sears to the district court on this issue. We need only highlight some significant findings of the court in support of our determination that the court's finding was not clearly erroneous. The court found that [t]he most credible and convincing evidence offered at trial regarding women's interest in commission sales at Sears was the detailed, uncontradicted testimony of numerous men and women who were Sears store managers, personnel managers and other officials, regarding their efforts to recruit women into commission sales. Id. at 1306. These witnesses testified to their only limited success in affirmative action efforts to persuade women to sell on commission, and testified that women were generally more interested in product lines like clothing, jewelry, and cosmetics that were usually sold on a noncommission basis, than they were in product lines involving commission selling like automotives, roofing, and furnaces. The contrary applied to men. Women were also less interested in outside sales which often required night calls on customers than were men, with the exception of selling custom draperies. Various reasons for women's lack of interest in commission selling included a fear or dislike of what they perceived as cut-throat competition, and increased pressure and risk associated with commission sales. Noncommission selling, on the other hand, was associated with more social contact and friendship, less pressure and less risk. This evidence was confirmed by a study of national surveys and polls from the mid-1930's through 1983 regarding the changing status of women in American society, from which a Sears' expert made conclusions regarding women's interest in commission selling; morale surveys of Sears employees, which the court found demonstrate[ ] that noncommission saleswomen were generally happier with their present jobs at Sears, and were much less likely than their male counterparts to be interested in other positions, such as commission sales, id. at 1310; a job interest survey taken at Sears in 1976; 17 a survey taken in 1982 of commission and noncommission salespeople at Sears regarding their attitudes, interests, and the personal beliefs and lifestyles of the employees, which the court concluded showed that noncommission salesmen were far more interested in commission sales than were noncommission saleswomen, id. at 1312; and national labor force data. 45 The court recognized the EEOC's expert witness testimony regarding women's general interests in employment, which essentially was that there were no significant differences between women and men regarding interests and career aspirations. We cannot determine the district court clearly erred in finding the evidence not credible, persuasive or probative. These expert witnesses used small samples of women who had taken traditional jobs when opportunities arose. Larger samples would have been more persuasive. In addition as the court found, [n]one of these witnesses had any specific knowledge of Sears. Id. at 1314. The court found Sears' evidence clearly more persuasive on the issue of different interest in commission selling between men and women. The court also found significant Sears' evidence that women became increasingly willing to accept commission sales positions between 1970 and 1980 due to, among other things, changes in commission sales positions from mostly full-time and largely part-time (more women preferred part-time), change in compensation to salary plus commission (which eliminated a lot of risk), increased availability of day care, and a group of successful saleswomen who served as role models. 46 The EEOC attacks the court's findings on interest on various grounds. We have already refuted the EEOC's basic argument that Sears' nonstatistical interest evidence is insufficient as a matter of law to successfully attack the EEOC's statistical case. The EEOC further argues that its analysis of the Applicant Interview Guides (AIGs) quantified interest and provided more persuasive evidence of interest than Sears' evidence on interest. According to the EEOC, the AIG analysis showed that while women were over 60% of full-time sales applicants and approximately 40% of the persons who considered themselves most ready, willing and able to sell installed home improvements jobs at Sears, women only comprised 1.7% of full-time commission sales hires in 1973 and between 10.5% and 5.3% thereafter. Sears argues in response that some of its evidence regarding interest, in the form of survey and labor force data, was indeed quantified. The district court gave little weight to the EEOC's AIG analysis--we discuss our decision that the district court's finding was not clearly erroneous in more detail at a later point. In short, the district court found that although neither the EEOC's nor Sears' analysis of the AIGs was entitled to much weight, Sears' analysis was more helpful on the question of differences in interest for commission selling among men and women. 47 The EEOC next argues that Sears maintained as part of its affirmative action program data on women refusing commission sales positions and that this would have provided an easy way of substantiating Sears' interest arguments, but Sears did not introduce any of this evidence. Sears had introduced exhibit 25, however, which was a collection of documents on refusals and indications of lack of interest in commission sales from the Eastern Territory. The EEOC notes that this contains only eight instances of female outside applicants refusing positions throughout the period at issue. The court nonetheless found that this exhibit corroborated Sears' witnesses' testimony that there were sincere and extensive efforts to recruit women into commission sales. Furthermore, we find the district court did not clearly err in crediting the testimony of Sears' store management witnesses regarding efforts to encourage women to take commission sales positions and women's lack of interest in those positions. 18 48 The EEOC bases another argument on the court's finding that although the surveys and testimony preserved by Sears related primarily to the interests of women already employed by Sears, the evidence was also a good indication of the interest of women applying for sales positions at Sears. Sears II, 628 F.Supp. at 1312 n. 56. The EEOC argues that there is no basis in the record for this inference that noncommission sales employees' interests were the same as those of women who applied for sales positions. But as we later discuss regarding the applicant pool, applicants' interests were not clear and the EEOC did not distinguish among applicants who were interested in commission as opposed to noncommission selling. Furthermore, the EEOC's own expert Siskin testified that fewer interested and qualified applicants for commission sales were to be expected in the applicant pool. 49 If it is argued that there was no interest in commission selling only because there were no opportunities, which disputes Sears' argument that there was instead an actual lack of interest in the existing opportunities, we are faced with the problem of which comes first, interest or opportunity, a chicken-egg problem. This is again an area where EEOC might have called a representative group of disappointed witnesses who preferred commission selling, but were rebuffed. It did not. 50 In short, we hold that the district court did not clearly err in finding that women were not as interested in commission sales positions as were men. 51 We similarly find that the district court did not clearly err in concluding that women applicants had different qualifications than did men applicants. The court noted that the EEOC's Commission Sales Report indicated that on average, female applicants in the 'sales' pool were younger, less educated, less likely to have commission sales experience, and less likely than male applicants to have prior work experience with the products sold on commission at Sears. Sears II, 628 F.Supp. at 1315. The EEOC does not challenge this finding. 52 All three of the court's findings discussed above--that commission selling is significantly different from noncommission selling, that women were not equally interested with men in commission selling at Sears, and that women applicants were not equally qualified with men for commission selling at Sears--form the bases for the court's criticisms of the EEOC's statistics regarding hiring at Sears.
53 The EEOC challenges the district court's characterization of its applicant pool, which was the source of the EEOC's hiring statistics, as inflated. The EEOC's expert Siskin constructed this applicant pool from the employment applications of 33,000 rejected sales applicants from 33 randomly selected Sears stores, and the applications of approximately 1,920 persons hired into full-time and part-time commission sales positions between 1973 and 1980 at approximately 210 stores. With the data Siskin attempted to determine the characteristics of male and female applicants by coding information from the applications onto computer tapes; in addition, the EEOC attempted to estimate the female proportion of sales applicants at all Sears stores on a nationwide and territorial basis. To identify all commission sales hires and promotions for 1973 through 1980, the EEOC also analyzed Sears' computerized payroll records for all Sears employees for those years. 19 See Sears II, 628 F.Supp. at 1294. 54 Using the payroll records, Siskin estimated the female proportion of full-time and part-time commission sales hires in the nation and each territory for each year from 1973 through 1980. To arrive at figures for the female proportion of full-time and part-time commission sales applicants, Siskin analyzed the sample of applications of nonhired applicants and counted as commission sales applicants all applicants who had applied for any job at Sears except those persons who requested only a nonsales job. The application, however, did not distinguish between commission and noncommission sales jobs. Siskin then compared the estimated percentages of women commission sales hires (actual percent female) with the percent of women in the sales applicant pool (expected percent female), on a nationwide and territorial basis for each year from 1973 through 1980. 55 The district court found that these comparisons resulted in large disparities between EEOC's actual and expected percent of female commission sales hires on a national and territorial basis for all years, with the z values, or number of standard deviations between the actual and expected figures, in a highly statistically significant range nationwide as well as in individual territories. 20 Sears II, 628 F.Supp. at 1295-96. 56 The court initially criticized the EEOC's applicant sales pool as inflated because Siskin included in the pool of applicants for sales positions anyone who had not checked nonsales jobs only. This meant that the EEOC considered as an applicant for a commission sales position anyone who had checked either sales, any of the above, or sales and another type of job (possibly including nonsales jobs). Furthermore, the court found that assuming arbitrarily that all members of the sales pool were applying for all commission sales positions at Sears in all divisions was one of the most serious flaws pervading all of EEOC's statistical analyses. Id. at 1301. We agree with the district court's conclusion that this led to an overinclusive sales pool, because it did not distinguish between commission selling and noncommission selling or account for differences in interests or qualifications among applicants. We also agree with the court's conclusion that the EEOC offered no credible evidence to support either the assumption that all applicants who indicated any interest in sales were specifically interested in commission sales or the assumption that all members of the sales pool were applying for all commission sales positions at Sears in all divisions, and that Sears' evidence proved that both assumptions were false. Id. at 1305. Based on the court's findings that women were differently qualified and interested in commission selling at Sears than were men, the court found women were disproportionately represented in the EEOC's commission sales applicant pool and thus were overestimated in the EEOC's expected percentage of women hires, which had been based on that applicant pool. The EEOC does not seriously challenge this conclusion, 21 and we do not find it is clearly erroneous.
57 The EEOC's basic argument is that the district court's criticism of its applicant pool is superfluous. The EEOC contends that its regression analyses took into account differences in applicant interest and qualifications and still showed significant disparities between the expected and actual percentages of female commission sales hires. The district court found a myriad of flaws in these analyses, however, and thus determined that they could not be given any weight. 58 We briefly summarize the regression analyses. Siskin performed various multiple regression analyses to determine if the disparities between expected and actual female percentage of commission sales hires could be explained by differences in characteristics among male and female applicants. 22 He chose six factors that he believed might affect an applicant's chance of being hired: (1) job applied for; (2) age; (3) education; (4) job type experience; (5) product line experience; and (6) commission product sales experience. Siskin coded information on each of these characteristics (from the applications) for each person in the hired and nonhired sample groups, and grouped each characteristic into various categories. He then performed various multiple regression analyses designed to demonstrate that discrimination rather than any of the six characteristics accounted for disparities in hiring of female applicants. The EEOC relied for the most part on the results of two regression analyses. In a weighted logit regression analysis, the EEOC measured the impact of each independent variable (e.g., age) on the dependent variable of whether or not the person is hired, controlling for the impact of other independent variables. In the multivariate cross-classification, or multicell analysis, Siskin's model allowed all the variables to interact, as there was a separate cell for each possible combination of all the different values for all of the independent variables, or characteristics. As the district court explained, this method of analysis calculates the expected percent female by comparing the number of hires with any given combination of characteristics, with the number of female non-hires with that particular combinations [sic] of characteristics. Sears II, 628 F.Supp. at 1297. 59 As the district court noted, the logit and multivariate analyses reduced the disparities between expected and actual female commission sales hires. The logit analysis reduced the nationwide expected proportion of female commission sales hires for 1973 through 1980 from 61.1% to 49.5% for full-time and from 66.2% to 63.3% for part-time. The multivariate analysis reduced the expected percentage of female full-time hires nationwide for all years from 61.1% to 40.3% (later reduced to 37.2%), 23 and the disparity for part-time hires was reduced from 66.2% to 60.5%. The court also noted that the z values were below 3 for a number of years in various territories and that the EEOC had admitted that those disparities were not statistically significant, but also recognized the EEOC's claim that statistically significant disparities existed nationwide and by territory for several years in which the z value exceeded 3. 24 60 The EEOC conducted another multivariate analysis for full-time hires for each of ten product line groupings (e.g., home furnishings, appliances) separately on a nationwide basis for the years 1973 through 1975, and 1976 through 1980. The court noted that there were statistically significant disparities in several of the product line groupings. 25 The EEOC conducted a similar analysis for the part-time case, except that fourteen instead of ten product line groupings were used and the Midwestern Territory was analyzed separately from the combination of the other territories because of apparent differences in hiring patterns. 26 61 The district court's major criticism of the EEOC's regression analyses is that the six characteristics chosen by the EEOC's expert Siskin were simplified and inadequate to account for applicant differences in interest and qualifications. Sears II, 628 F.Supp. at 1302, 1305 n. 35a. The EEOC contends that these characteristics did indeed contemplate differences in interest and qualifications. The court first criticized the EEOC's choice of characteristics. The court found, and the EEOC does not challenge this, that the six factors chosen by Siskin were the result of his opinion as to what factors would be important for commission selling. The court further found it could give little weight to Siskin's opinion in this regard because Siskin is not an expert in labor economics, has no expertise in the area of retail sales, and has no direct knowledge of Sears. Id. at 1302. We must accord great deference to such credibility-based factual determinations. See Anderson v. City of Bessemer City, 470 U.S. 564, 575, 105 S.Ct. 1504, 1512, 84 L.Ed.2d 518 (1985). 62 The court also determined that the EEOC presented no credible evidence to support Siskin's choice of the particular six factors as important to commission selling at Sears. Instead, the court found all the evidence offered at trial indicates ... that a number of important factors were not included in the EEOC's analysis. Sears II, 628 F.Supp. at 1302. The most critical factor intentionally excluded by EEOC according to the court was the applicant's interest in commission sales and in the product to be sold. Id. The EEOC contends on appeal that four of its characteristics, Job-Applied-For and the three experience variables (job type experience, product line experience, and commission product sales experience), were highly correlated with interest. It cites nothing, however, either on the record or outside the record, to support this statement. The case is thus unlike that of Bazemore v. Friday, 478 U.S. 385, 106 S.Ct. 3000, 3008, 92 L.Ed.2d 315 (1986), in which the Supreme Court noted that the type of independent variables used in plaintiffs' regression analyses were selected based on discovery testimony by one of defendants' officials that certain factors were determinative of the dependent variable salary. Although this rationale for choice of the independent variables did not preclude questioning of the comprehensiveness of these variables, see Bazemore, 106 S.Ct. at 3008-09, the Court rejected the court of appeals' conclusion that the analyses were unacceptable because they did not include all variables that might have an effect on salary--the Court stated that the analyses should be given some probative value. We think that the EEOC's failure to support its choice of variables in this case casts a shadow on the probative value of the regression analyses incorporating those variables. Furthermore, if those four variables were indeed highly correlated with interest, the EEOC still does not explain, satisfactorily, how the analyses incorporating those variables adequately controlled for interest. 63 Interest was not the only factor affecting an applicant's chance of being hired that the court found was not controlled for in the EEOC's regression analyses. The EEOC challenges the district court's conclusion that its regression analyses did not adequately adjust for differences in qualifications. As the district court found, the EEOC's Commission Sales Report itself demonstrated that, on average, female applicants in the 'sales' pool were younger, less educated, less likely to have commission sales experience, and less likely than male applicants to have prior work experience with the products sold on commission at Sears. Sears II, 628 F.Supp. at 1315. The EEOC counters that its experience variables took those differences into account. The EEOC's three experience variables would seem to account for differences in the experience aspect of qualifications. The court found that other characteristics, however, such as physical appearance, assertiveness, the ability to communicate, friendliness, and economic motivation, factors Sears managers had identified as desirable for commission salespersons, could be determined only from an interview, not from the written application. Id. at 1303. The court recognized that these characteristics are not easily quantified and this data would not be generally available from the application forms, but found that to the extent the EEOC's regression analyses did not incorporate these factors, they were entitled to less weight. We cannot say that this finding, going to the probative value rather than the relevancy of the regression analyses, is clearly erroneous. See Bazemore, 106 S.Ct. at 3008-09. 64 We believe the district court accorded its findings regarding the failure to include independent variables such as interest and aspects of qualifications in the regression analyses the proper evidentiary weight. The district court cited the Fourth Circuit case Bazemore v. Friday, 751 F.2d 662, 672 (4th Cir.1984), aff'd in part and rev'd in part, 478 U.S. 385, 106 S.Ct. 3000, 92 L.Ed.2d 315 (1986), for the principle that in a meaningful regression analysis, all independent variables that significantly affect the dependent variable should be included. The Supreme Court later reversed the Fourth Circuit on this issue, holding that the Court of Appeals erred in determining that a regression analysis was  'unacceptable'  because it did not include  'all measurable variables'  that could affect the dependent variable. Bazemore, 106 S.Ct. at 3009 (quoting opinion of court of appeals). The Court noted that the regression analysis accounted for the major factors and in that situation failure to include variables normally goes to probativeness, not admissibility. Id. The district court in this case did not make the Fourth Circuit's fatal mistake. It is clear from the district court's opinion that it did not determine that the EEOC's analyses were inadmissible or irrelevant because of the failure to include variables--instead, the court determined that the analyses were not probative because of the omissions. Although we do not believe the court found the regression analyses totally unacceptable, the court's criticisms of the analyses are so pervasive that the court may have believed that the regressions were what the Bazemore Court termed so incomplete as to be inadmissible as irrelevant, which would be an exception to the general rule established in Bazemore. 27 Bazemore, 106 S.Ct. at 3009 n. 10. 65 The district court's other criticisms of the regression analyses go to the coding of the factors the EEOC did include in the regressions. The court found that the coding of several factors was clearly inadequate to properly adjust for those factors. Sears II, 628 F.Supp. at 1303. The court noted that regarding the characteristic of prior job experience, the EEOC did not adjust for various forms of job experience. The court gave the example of the prior job experience of commission selling--women were more likely than men to have been cashiers on a commission-selling basis, and thus in actuality did not have experience in commission selling as it was done at Sears. The EEOC responds that its expert Siskin performed a multicell regression separating out all persons who had been cashiers with trivial impact on the results. 28 Although cashiers may not have been the best example of earlier job experience, we do not believe the court's emphasis on prior job experience was implausible on this record. The district court found other infirmities in the coding of the experience variables; namely, that Siskin did not code the amount of prior experience, interruptions in work history, or other relevant experience (the latter of which was on the application). These inadequacies in coding of the experience variables led the court to conclude that [b]ecause of the nature of these coding deficiencies, women are credited with greater amounts of experience at higher levels than they actually had. The effect of all these inadequacies is to inflate the percent of women the statistical models estimate should be hired. Id. at 1304. The EEOC responds to some, but not all of these criticisms, mainly relying on its argument that the court did not cite actual data to support the criticisms. We do not believe the court's criticisms of the coding of experience variables are clearly erroneous. 66 We agree with the EEOC that one of the district court's criticisms of the coding was misplaced. The court concluded that Siskin had not recoded the Job-Applied-For variable after discovering an error in the coding, and that there was thus an overestimation of the expected female proportion of commission sales hires. Siskin had, however, recorded the variable for the overlapping sample of twenty-nine stores and reported in court that there was no impact on the analysis. 67 The EEOC argues that none of the court's other criticisms of its regression analyses are of consequence. The district court found that by creating artificial national and territorial pools of applicants, without taking into account likely variations in the quality of applicants at individual stores or even whether openings actually existed at stores, the EEOC did not analyze the hiring situations actually confronted by Sears managers. Id. at 1316. The court thus accorded the statistical analyses less weight. We believe that on this record such a finding is plausible. 68 Based on the above findings, the district court determined that the z values both for the unadjusted comparisons of expected and actual female hires, and for the regression analyses, did not constitute statistically significant disparities. Regarding the unadjusted analysis, the court found that because the EEOC never controlled for interest or qualifications, there were no meaningful comparisons between actual and expected female hires and thus the z values prove[d] nothing. Id. The court also found that the EEOC's attempts to adjust for differences among applicants, mainly interest and qualifications, were woefully inadequate as discussed above, so that the EEOC did not achieve fair estimates of the expected female hires. In such a case, the court concluded, the z statistics connected with the comparison of actual and expected female hires have little value. Id. The court further credited Sears' incorporation of variances into calculation of the z statistics to account for the difference between the proportion of women in the nationwide pool and the proportion of women in each store pool. After taking into account the variances, the court noted, the z statistics were significantly lower, so that [f]or EEOC's nationwide multicell analysis of full time commission sales hires, z values exceed 3 in only two years. 29 Id. at 1317. The court then concluded that because these z values did not take into account differences in male and female interest in commission sales, an analysis incorporating those differences would not produce statistically significant disparities. 69 The EEOC challenges the above findings, arguing that its regression analyses did indeed show statistically significant disparities that satisfied its burden of proving discrimination as articulated by the Supreme Court in Bazemore, 106 S.Ct. at 3009: A plaintiff in a Title VII suit need not prove discrimination with scientific certainty; rather, his or her burden is to prove discrimination by a preponderance of the evidence. The Bazemore Court went on to note, however, that [w]hether, in fact, such a regression analysis does carry the plaintiffs' ultimate burden will depend in a given case on the factual context of each case in light of all the evidence presented by both the plaintiff and the defendant. Id. We have not as yet discussed all of the EEOC's evidence, but we need only briefly consider that other evidence because the EEOC's primary focus was its regression analyses and with one exception (the AIGs), the EEOC casts that evidence as supporting its regression analyses in allegedly showing hiring discrimination. We believe the district court's treatment of the EEOC's regression analyses was consistent with Bazemore --the court did not require the EEOC to show disparities with scientific certainty, but rather considered all the evidence in determining that it could not rely on the EEOC's purported disparities.
70 To support the results of the regression analyses, the EEOC statistically analyzed responses to Sears' Applicant Interview Guides (AIGs) that had been used in certain stores in the Denver, Colorado, area in 1978-1980, and in Waco, Texas, in 1980. Siskin analyzed these applicants' ratings, found in the Guides, on a scale of one to five of their own interests and experience with regard to four of forty-five categories chosen by Siskin as closely approximating four product line groups at Sears: appliances, automotive, home building materials, and home improvements. This analysis did not incorporate any of the characteristics used in the other regression analyses, and was completely independent from those analyses. The district court noted that Siskin estimated that the expected proportion of female commission sales hires was higher based on this analysis than it was based on the multivariate regression analysis. 71 The EEOC's main argument is that its AIG analysis supports the regression analyses. It also implies, however, that even considered on its own, the AIG analysis proves Sears' intentional hiring discrimination. We reject both contentions. The district court explicitly stated that he ascribed little weight to the AIG analyses, whether analyzed by the EEOC or Sears, and we agree with this conclusion. The AIGs have a very limited scope, coming from only two stores in Sears' Southwest Territory. In addition, the Guides at one store were administered only in 1978, while in the other they were administered from 1978 to 1980. The Guides simply cannot serve as a basis for proving the EEOC's claim of nationwide discrimination for the years 1973 to 1980. 72 The EEOC nonetheless claims that the AIGs were corroborative of its multicell analyses because after the EEOC adjusted its sales pool using the interest and experience responses on the AIGs, there were statistically significant disparities. The court found that this analysis was tainted because of the flawed sales pool, which we discussed above, and because the adjustments were incomplete, not taking into account age and education or more than just a few product line groupings. While we may not agree that factors such as age were crucial to this analysis, we cannot find that the court's conclusion regarding the value of this evidence, especially considering its limited scope, was clearly erroneous. 73 Furthermore, it is apparent that the district court found Sears' analysis relatively more helpful regarding the separate issue of differences in male and female interest in commission sales. The court found that this evidence, which showed that men were more interested in jobs usually associated with commission sales positions at Sears while women were disproportionately interested in noncommission-type jobs, corroborated Sears' survey evidence regarding differences in male and female interests. Sears' expert Dr. Joan Haworth had analyzed the AIGs by normalizing the scores that applicants gave themselves in interest, skill and experience, considering that some applicants might inflate their scores to increase their chances of being hired. Haworth, like Siskin, chose categories roughly corresponding with Sears' product lines and adjusted the EEOC's expected percentage of women commission sales hires based on normalized responses in the Guides. She also adjusted the expected percentage based on figures indicating that one of every 250 job applicants is selected for commission sales. After these adjustments, disparities were greatly reduced for the period of 1976 to 1980. The EEOC criticizes the validity of Haworth's analysis on several grounds and argues that the district court gave the analysis too much weight, especially in light of the court's criticisms of the EEOC's AIG analysis. We disagree. The court noted various deficiencies in Haworth's analysis, explicitly stated it had not given it substantial weight, and it is clear that the weight the court did accord this evidence was not in the more specific sense of Haworth's product line comparisons, which was the counterpoint to the EEOC's analysis, but rather regarding women's relative lack of interest in commission selling at Sears. Sears II, 628 F.Supp. at 1323-24. We cannot say the district court clearly erred in attributing to this evidence the limited significance that it did.
74 The EEOC claims that the external labor force data introduced by Sears on the issue of differences in male and female interest, which the court found corroborate[d] Sears' strong evidence of female lack of interest in commission sales and in ... particular product lines, id. at 1314, instead supports the EEOC's claim that Sears discriminated against women in hiring. The EEOC cites various nationwide statistics for female percentages of salespeople in various product lines and compares those figures with the smaller figures at Sears. Sears responds that these comparisons are inaccurate because the nationwide data does not reflect the breakdown of noncommission versus commission salespersons depending on product line existing at Sears. According to Sears, the product line data included vast numbers of department store clerks who did not sell on commission and whose jobs in no way resembled those of commission salespeople at Sears, and other data for retail commission salespersons failed to indicate product line differences so that it included saleswomen selling items like fashions and furs which are seldom sold on commission on Sears. It is unclear whether the EEOC actually presented this argument to the district court. Indeed, EEOC expert Siskin rejected reliance on external labor force data in constructing his analyses. Issues not presented in district court are waived on appeal. See DeValk Lincoln Mercury v. Ford Motor Co., 811 F.2d 326, 338 (7th Cir.1987). In any event, considering the inaccuracies likely present in a comparison of the nationwide data with Sears data on this particular question, we cannot find that the district court erred in not concluding that external labor force data indicated hiring discrimination by Sears. 75 The EEOC attacks the court's conclusion that certain nationwide external labor force data for all employed commission salespersons reflected a lower female availability for commission sales positions during the years 1976 to 1980 than the EEOC estimated were available based on its sales pool. The EEOC argues that it is inappropriate to compare a nationwide static work force figure with Sears' hiring figures for various years. We agree that such a comparison can be misleading. As noted by this court in Movement for Opportunity & Equality v. General Motors Corp., 622 F.2d 1235, 1345 (7th Cir.1980), such cumulative snapshot statistics include hiring decisions for the past twenty to thirty years. Thus the statistics mask past discriminatory hiring decisions and should not be compared with female availability for commission sales positions during a recent fouryear period. See id.; United States v. City of Chicago, 549 F.2d 415, 435 (7th Cir.1977) (it is no defense to a charge of discrimination that not everyone else is in compliance with the law). 76 Nonetheless, we believe that the main purpose of this evidence according to the district court was not to show overall female availability for commission sales positions, but rather to further corroborate differences in male and female interest by product line. The court found that this national labor force data did corroborate female lack of interest in selling particular product lines, and we cannot say that this finding is clearly erroneous. To the extent that the usefulness of this static labor force evidence is subject to the same criticism of incorporating past discrimination, which we find doubtful when the evidence is used for the purpose of showing differences in men's and women's interest in particular product lines, we note that some of the evidence that the court found most significant related to comparisons of male- and female-owned businesses, which would not seem to be as subject to discrimination.
77 The EEOC maintains that the district court erred in determining that none of its evidence regarding the subjective nature of Sears' hiring process or regarding Sears' testing practices supported the EEOC's statistical analyses or provided any credible proof of discrimination by Sears. With respect to the subjectivity of Sears' hiring process, the EEOC argued that there was no formal training and a lack of written instructions for Sears' interviewers regarding the qualities to look for in commission sales applicants. In addition, the EEOC pointed to what it referred to as a highly masculine description of a commission salesperson found in the Retail Testing Manual, which included phrases such as active, has a lot of drive, possesses considerable vigor, and likes work which requires physical energy. The EEOC also notes that a Sears official admitted that this reflected characteristics that on average more men possess than women. The EEOC cites numerous cases for the proposition that a lack of objective standards regarding employment decisions can be a discriminatory practice because such a system with its lack of safeguards may easily be subject to abuse. See Davis v. Weidner, 596 F.2d 726, 732 (7th Cir.1979) (court noted absence or extreme generality of any predetermined standards for selecting employees, the failure to record evaluations of job applicants, and the absence of minorities on the employment decisionmaking bodies can add credence of the claim of an aggrieved employee, especially when the employer does not rely on objective, easily measured criteria for his employment decision); Stewart v. General Motors Corp., 542 F.2d 445, 450-51 (7th Cir.1976) (court found discriminatory promotion practices where a process was highly subjective and loosely structured, the supervisory recommendations being important but there were no safeguards and no written guidelines regarding promotion criteria), cert. denied, 433 U.S. 919, 97 S.Ct. 2995, 53 L.Ed.2d 1105 (1977); see also Carmichael v. Birmingham Saw Works, 738 F.2d 1126, 1131 (11th Cir.1984) (subjective procedures in context of promotion included word of mouth regarding openings and informal review procedures); Harris v. Birmingham Board of Education, 712 F.2d 1377, 1383 (11th Cir.1983) (lack of objective hiring standards contributed to [plaintiff's] establishment of a prima facie case of racial discrimination in hiring); Williams v. Colorado Springs School District, 641 F.2d 835, 842 (10th Cir.1981) (observing that subjective promotion and employment criteria afford possibilities of abuse); Rowe v. General Motors Corp., 457 F.2d 348, 358-59 (5th Cir.1972) (discriminatory promotion/transfer procedures where foreman's recommendation is most important factor in promotion process, foremen are given no written instructions regarding qualifications necessary for promotion, controlling standards were vague and subjective, hourly employees not notified of promotion opportunities and no safeguards). 78 Sears concedes that its hiring process was subjective, but argues that subjectivity is a necessary part of the hiring process and it does not follow that because a hiring process is subjective, it is necessarily discriminatory. Cf. Movement for Opportunity & Equality v. General Motors Corp., 622 F.2d 1235, 1277 (7th Cir.1980) (court noted that subjectivity was inherent in promotion system, but evidence established that system contained checks and balances and, in addition, the promotion system included procedures to fulfill affirmative action obligations); Mozee v. Jeffboat, Inc., 746 F.2d 365, 371, 372 (7th Cir.1984) (subjective and other intangible factors may influence employment decisions and ... even subjective misjudgments may not necessarily be the basis for Title VII liability, in addition, the probative value of statistical evidence may be diminished where an employer genuinely relies on subjective factors in making decisions and where data are incomparable (emphasis in original)); Williams v. Colorado Springs School District, 641 F.2d 835, 842 (10th Cir.1981) (subjective definitions of employment selection and promotion criteria not unlawful per se, because some subjectivity cannot realistically be avoided in making hiring and promotions decisions). Sears argues that subjectivity is especially important in its affirmative action efforts. Cf. Movement for Opportunity & Equality, 622 F.2d at 1277. 79 The question, then, is how much subjectivity is permissible. We agree with the Ninth Circuit that a court must consider an employer's use of subjective criteria ... with the other facts and circumstances of the case. Casillas v. United States Navy, 735 F.2d 338, 345 (9th Cir.1984). While it may have been better policy for Sears to have had written instructions and formal training for interviewers regarding qualities to look for in commission sales applicants, we cannot say on this record that Sears exercised so much subjectivity as to engage in a discriminatory practice. In addition, the court's finding regarding Sears' affirmative action efforts appears to obviate impermissible subjectivity on the part of Sears' interviewers. 80 Regarding the description of commission salespersons found in the Retail Testing Manual, the EEOC argues that this instruction was reflected in oral instructions to interviewers. The district court found that there was no evidence that this historical description had any influence on hiring decisions, particularly during the years in question, that Sears managers often paid no attention to information in the manual, and that [t]here is no basis in the record for concluding that this description had any negative impact on the selection of women for commission sales. Sears II, 628 F.Supp. at 1318. After consulting the record, we conclude that the court did not clearly err in this finding. 81 The EEOC also argues that the district court erred in finding that one aspect of Sears' testing practices was not discriminatory. The EEOC contends that a vigor scale, one of seven scales on the Thurstone Temperament Schedule, contained questions that would more likely be answered affirmatively by men, such as Have you played on a football team? The district court agreed that these questions would more likely be answered affirmatively by men, but chose to believe Sears' witnesses, who testified that many Sears applicants were not tested, or if they were tested, it was not until after they were hired. The court also believed Sears' managers, who testified that the test had little impact on any decision to hire and that test scores were adjusted for women. The EEOC argues that it is implicit in the district court's phrasing of the impact of the test that it had at least some impact. In light of the court's conclusion that there was no credible evidence that a woman's vigor score ever prevented her from being hired for commission sales at Sears, however, we find the court did not clearly err in determining this aspect of Sears' testing did not have enough impact, on its own, to prove a case of discrimination.
82 The EEOC contends that the court erred in not mentioning hiring data for 1972 because, although it was before the liability period began, it would constitute evidence of past discrimination. The EEOC cites Bazemore v. Friday, 478 U.S. 385, 106 S.Ct. 3000, 3010, 92 L.Ed.2d 315 (1986), in which the Supreme Court stated that  '[p]roof that an employer engaged in racial discrimination prior to the effective date of Title VII might in some circumstances support the inference that such discrimination continued, particularly where relevant aspects of the decisionmaking process had undergone little change.'  (quoting Hazelwood School District v. United States, 433 U.S. 299, 309-10 n. 15, 97 S.Ct. 2736, 2742-43 n. 15, 53 L.Ed.2d 768 (1977)). 83 The EEOC cannot prevail with this argument, however, because based on the district court's findings, which we have held are not clearly erroneous, it has not proved that Sears discriminated against women in hiring in 1972. The EEOC cites various statistics that are products of the same analyses we have described above, and argues that they show discrimination. These analyses, however, are subject to the same defects the district court found fatal to the statistics of later years. Furthermore, these statistics may likely be even less reliable than those of the later years. The EEOC's expert Siskin testified that these 1972 figures were at best an estimate. Because we cannot say that these statistics provide proof of past discrimination, we need not determine whether the statistics support an inference of later discrimination.
84 According to the district court, the EEOC relied almost exclusively on statistical analyses to prove its claim that Sears discriminated against women in promotions from noncommission sales jobs to commission sales jobs. 30 Sears II, 628 F.Supp. at 1300. The EEOC's expert Siskin estimated the expected proportion of female promotions for each year by looking to the female proportion of noncommission salespersons in each store at the end of a year (year-end store pool). He compared these figures with the proportion of women actually promoted into commission sales, and came up with z values that exceeded 3 in all but one year in one territory. To account for the possibility that persons in a division in which a promotion occurred would have a greater chance at the promotion than a salesperson outside the division, Siskin performed a second analysis of this data. He estimated the expected female proportion of promotions by looking to the female proportion of noncommission salespersons on the basis of division (year-end division pool). The z values for this analysis exceeded 3 in all years and territories except for one territory in one year. 31 85 As it did with the EEOC's hiring statistics, the court criticized the EEOC's promotion statistics at two levels: (1) the statistics were based on the false assumptions that all those noncommission sales employees, who composed the pool for promotions, were equally qualified and that they were equally interested in promotions from noncommission to commission selling; (2) even if the EEOC's promotion statistics were valid, the disparities created by those statistics were significantly reduced or even eliminated after Sears' adjustments for differences in interest, and other adjustments. 86 We do not think that the burden initially was on the EEOC to adjust its promotion statistics for interest and qualifications. After Sears introduced data regarding interest and qualifications, however, the EEOC used a careful litigation strategy in attempting to show that interest and qualifications would have little bearing on its disparities, or at least would not eliminate them. The question is whether the EEOC's disparities remained statistically significant after taking into account adjustments for differences in men's and women's interest in commission selling, for differences in qualifications, and other considerations (such as affirmative action and the lack of individual victim testimony). 87 Initially, we note that the court gave some credence to a Sears Product Line adjustment of the EEOC's promotion analysis that resulted in reducing the expected female proportions of promotions from noncommission to commission sales from EEOC's division pool analysis for the 1974-1980 period from 68.6% for full-time and 73.8% for part-time to 60.8% for full-time and 64.5% for part-time. That adjustment was not even entitled to the little credence the court gave it, however, because this product line adjustment was subject to the same criticisms the court made of Sears' Division Feeder Pool adjustment of the promotion data. 32 We therefore assign this adjustment no weight in determining whether the district court erred in determining that Sears successfully produced probative evidence to undermine the EEOC's promotion disparities. The district court recognized that this analysis tended to overreduce the EEOC's disparities, and apparently only gave the adjustment weight for the principle that adjustments could reduce the disparities.
88 Sears' general evidence regarding differences in men's and women's interest in commission selling that we discussed in the context of the hiring claim applies to the promotion claim as well. Sears also adjusted the EEOC's promotion data based on interest measurements. The district court concluded that Sears' analysis of some of that evidence demonstrates that interest alone can account for the disparities computed under EEOC's analysis. Sears II, 628 F.Supp. at 1326. The EEOC contends on appeal, however, that it did take into account major adjustments for interest and statistically significant disparities still remained for both part-time and full-time throughout the period at issue. 89 After analyzing the record regarding both parties' evidence concerning interest and promotions, we conclude that the district court did not clearly err in determining that Sears' interest evidence substantially reduced (and indeed almost eliminated) the EEOC's alleged promotion disparities. 90 We first address the EEOC's contention that it did take into account major adjustments for interest and that significant disparities remained after those adjustments. The EEOC points us to two purported interest adjustments. In the first adjustment, the EEOC picked two years, 1973 and 1977, and estimated the expected proportion of female promotions if noncommission salesmen were 1.5, 2, or 3 times as interested in commission sales than were noncommission saleswomen. For the 1973 figures, the expected proportion of female promotions was reduced, but it was still greater than the actual proportion at all three interest levels. For the 1977 figures, disparities existed except when the proportion of noncommission salesmen was three times the proportion of noncommission saleswomen--then the disparities between the proportions of expected and actual female promotions were eliminated. 91 These adjustments are meaningless in a vacuum, however. The EEOC does not point to any evidence on which it based its assumption that the proportion of interested noncommission salesmen was only 1.5, 2, or 3 times as great as the proportion of interested noncommission saleswomen. The district court might well have concluded, based on Sears' evidence regarding differences in male and female interest, that more than three times the proportion of noncommission salesmen as of noncommission saleswomen was interested in commission selling. Indeed, this adjustment seems probative of Sears' argument that differences in interest can account for disparities. 92 The EEOC's other interest adjustment at least was based on some quantified data regarding different interest levels. The EEOC as well as Sears relied on responses to the 1982 National Timecard Nonsupervisory Special Survey (NTNSS), which was administered to incumbent noncommission and commission salespeople by Sears in 1982. The EEOC's expert Siskin analyzed responses to question 13C of the NTNSS 33 to achieve quantified levels of interest and came up with a 1.78 ratio of male to female full-time noncommission interest in commission selling, and a 1.56 ratio of male to female part-time interest. Siskin adjusted the original expected female proportions of promotions based on these interest responses and found that the disparities were significantly reduced, although the EEOC argues that statistically significant disparities remained. 34 93 The EEOC's NTNSS interest adjustment reduced the disparities between expected and actual proportions of female promotions by one-half. If the adjustment were considered valid, the EEOC is correct in arguing that there would nonetheless remain disparities that are above the statistically significant z level of 3 for both full-time and part-time promotions. For a variety of reasons, however, we think the district court did not clearly err in not crediting this interest adjustment by the EEOC and thus not determining that statistically significant disparities remained even after taking interest into account. 94 First, the NTNSS was administered in 1982, but the EEOC applied that 1982 interest ratio to each year and all years from 1973 to 1980. The district court found, however, and we have previously determined that finding was not clearly erroneous, that women's interests changed substantially over the decade of the 1970's. It is significant that after the EEOC's interest adjustment for full-time promotions, the significance of the disparities declined during the 1970's so that in 1979 the disparity was 2.9 (less than statistically significant), and in 1980, the z level was 3.6 (barely statistically significant). The district court noted women's changing interests in addressing some of Sears' interest adjustments, finding it important that even in 1982-83, noncommission salesmen were three times as likely as noncommission saleswomen to be interested in moving to commission sales. Sears II, 628 F.Supp. at 1326 (emphasis added). 95 It is also a bit incongruous that the EEOC chooses to rely on the NTNSS interest adjustment on appeal, because in district court, the EEOC criticized the reliability of the responses to that survey. The EEOC argued that noncommission salespeople could not adequately be identified from responses to the survey, and that the effect of that imprecision was to reduce the expected female proportion of promotions after adjustments for interest. The district court recognized that although there were some flaws in the design and administration of the NTNSS, they are not significant enough to undermine the essential validity of the results, Sears II, 628 F.Supp. at 1312, and also determined that Sears was adequately able to identify noncommission salespersons. It would seem that any imprecision would also tend to cut against the EEOC if the EEOC is relying on these statistics. 96 Perhaps the biggest problem with the EEOC's interest adjustment is that it relied on responses to question 13C of the NTNSS, whereas the district court found that question 42 of that survey, 35 the responses to which Sears relied on in its interest adjustment, was a better indicator of real interest in commission selling. The court determined that question 42 was a better measure of interest because in answering that question, the respondent had to affirmatively choose commission selling rather than other forms of selling. Question 13C, on the other hand, merely required the respondent to answer whether he or she would accept a commission sales position if it were offered. Sears' interest adjustment of the EEOC's promotion data based on responses to question 42 reduced the expected female proportion of promotions so as to virtually eliminate any disparities between expected and actual female proportion of promotions in both the full-time and part-time cases for the all years period of 1974 to 1980. 36 97 The EEOC argues that the question 42 interest adjustment still results in statistically significant disparities in full-time and part-time 1973 promotions (the z levels were 6.1 for full-time and 11.5 for part-time in 1973). We do not think that the court erred in not reaching the conclusion suggested by the EEOC. As we noted above, women's interests changed over the 1970's. It is therefore questionable to apply 1982 interest levels to 1973 promotion statistics. Sears' expert Haworth specifically stated changing women's interest levels as one reason why she had not applied the 1982 data to promotion statistics for the earliest years at issue. Furthermore, both Haworth and the EEOC's expert Siskin acknowledged that data underlying the 1973 promotion statistics were unreliable. 37 98 The EEOC also contends that the court erred in not noting the implications of a combined analysis based on responses to questions 13C and question 42. The EEOC notes that the court approved a Sears' witness's suggestion that perhaps the best measurement of interest would be achieved by combining responses to questions 13C and 42. The EEOC then presents an adjustment of its Division Pool results based on this combined analysis. Although it may have been helpful to have had the benefit of this combined analysis, we may not consider these statistics on appeal because they were not presented to the district court. The EEOC contends that we may consider this adjustment because the district court accepted the Sears' witness's testimony that this analysis would be the best method, and that the EEOC has done the analysis through simple mathematics that is fully explained. Even the results of simple math are inappropriate for this court to consider on appeal, however, if the results were not initially presented to the district court. See Ohio-Sealy Mattress Manufacturing Co. v. Sealy Inc., 776 F.2d 646, 650 n. 1 (7th Cir.1985) (rejecting plaintiff's argument that because summaries were simply tools to aid [the] court in understanding the record, court of appeals could consider summaries of voluminous materials although they had not been presented to the district court); Rebuck v. Vogel, 713 F.2d 484, 486 (8th Cir.1983); Fed.R.App.P. 10(a). Because the EEOC's combined analysis resulted in new percentages not presented to the district court, we may not consider that analysis on appeal. 99 Perhaps because the district court recognized some imprecision in the NTNSS survey responses, the court chose to rely on the evidence of interest expressed in responses to Career Aspiration Questionnaires (CAQs) administered by Sears at stores in the EEOC's nonhired applicant sample in 1982-1983. The court concluded that Sears' analysis of these questionnaires demonstrates that interest alone can account for the disparities computed under EEOC's analysis. Sears II, 628 F.Supp. at 1326. The EEOC attacks the court's reliance on and conclusions from these questionnaires in several respects. Although we find some troubling aspects of reliance on this data, 38 we do not agree that the district court's conclusion that adjustments for interest can virtually eliminate the EEOC's disparities is clearly erroneous. 100 The EEOC contends that the results from the CAQs are unreliable because Sears' expert Haworth combined full-time with part-time responses regarding interest in commission sales to achieve the results that the proportion of men interested in commission sales was approximately three times the proportion of women interested in commission sales. The district court recognized this criticism but determined that there is no indication that the part time responses do not fairly reflect female interest in commission sales. Id. We do not believe that this reasoning is faulty. The EEOC argues that career aspirations of full-time and part-time salespersons differ and product lines in which those persons work are usually distinct. Commission sales positions at Sears are not available only to full-time salespersons, however, thus the question of interest in commission sales seems to cut across the line between full-time and part-time. 101 The EEOC also suggests that the district court relied on the results of the CAQs and not the results from the NTNSS because the results of the NTNSS did not totally eliminate the promotion disparities while the results of the CAQs did. As we noted earlier, the court questioned the reliability of the NTNSS on several grounds, and even the EEOC questioned the reliability of the NTNSS. Furthermore, the court did not discount the results of the NTNSS; the court instead found that the results from the NTNSS were similar to and corroborated the results of the CAQs. It is clear that the court did not place sole reliance on the CAQs as the one interest survey that eliminated promotion disparities. The court found that two other surveys, the 1976 Job Interest Survey and the NTNSS, although they involved some reliability problems, yielded similar results on the question of interest. 39 Furthermore, the court expressed that several factors aside from interest caused it to determine that the EEOC's promotion disparities did not prove a case of intentional discrimination. Those factors, which we discuss in more detail later, include differences in qualifications between men and women, Sears' affirmative action efforts, and a lack of individual victim testimony. 102 The EEOC also questions the reliability of the CAQs based on the sample size of the CAQ responses. The CAQs were distributed to 1,220 female and 357 male full-time and part-time noncommission salespersons. Of that number, there were fifty responses of an interest in commission sales, four of which were from full-time noncommission salespersons. 40 Although it would have been helpful to have had more responses to analyze, we are not convinced that the small number of responses to this question are not indicative of interest in commission sales, especially in light of the significant numbers of respondents (63.4% of women and 50.1% of men) who answered no to the question of whether they were interested in transfer or promotion. This question immediately preceded and headed the question regarding interest in commission sales. See D. Baldus & J. Cole, Statistical Proof of Discrimination Sec. 4.123, at 53 (Supp.1986) (Sample size, of course, affects the validity of actual applicant data, but this should not lead the court to disregard it altogether. Even a small sample of applicant flow data sheds some light on the [employer's] actual operation.). 103 The EEOC argues that applying the CAQ male to female interest ratio of 3.19 still results in disparities in 1973. As we noted above regarding applying NTNSS survey results from 1982 to the 1973 promotion data, it is questionable to apply results of 1982 or 1983 survey data to 1973 statistics because women's interests had changed during the 1970's. In addition, the 1973 promotion statistics were based on unreliable promotion data. 104 Even considering the reliability problems of some of the interest survey data, we think the court did not err in determining that the interest evidence significantly, if not almost totally, reduced the promotion disparities.
105 Regarding the issue of differences in qualifications among male and female noncommission salespersons, the district court concluded that all evidence regarding qualifications shows that males at Sears were more qualified for commission sales positions than [were] females. Id. The EEOC complains that the court did not state what evidence it purported to rely upon in reaching this conclusion, and assumes along with Sears that the court was referring to noncommission salespersons' experience and background before entering noncommission sales at Sears. Both parties dig into the record for evidence to refute or support the proposition that prior experience can have an impact on chances of promotion from noncommission sales to commission sales. 106 Prior experience is only one aspect of qualifications, however, and we believe the district court was referring to the performance aspect of a noncommission salesperson's qualifications for a promotion to commission sales. In discussing Sears' data on sales performance in the hiring context, the court also referred to Sears' data on sales performance by commission salespersons during their first year after promotion. The court found that among full-time and part-time commission salespeople, the sales performance rates of the males exceeded the rates of the females. The court determined that the men Sears hired and promoted into commission sales were more successful salespeople, and inferred that they had more of the true qualifications for commission sales than did the women. The court found this true for all new hires and promotions, among the top 10% of the commissions salesmen and saleswomen, and among the bottom 10% of commission salespersons. 107 The EEOC, in response to the performance issue in the hiring context, states that only the bottom 10% of the hires or promotions are important to a discrimination claim because if Sears did discriminate against women, it was to be expected that women still should have lower performance rates than men on average and in the top 10%. Sears cites expert witness testimony in claiming that it is not to be expected that women would have lower performance rates than men. 108 It is true that in their first year after being promoted, men outperformed women each year from 1974 to 1980 in the full-time case overall, and in the top 10% of salespersons. In the bottom 10%, the performance of men and women was relatively equal, although men still outperformed women slightly. In the part-time case, there was a smaller sample, but men clearly outperformed women in the top 10% and usually outperformed women overall. The performance rates in the part-time case were about the same. Based on this data, we cannot say that the judge clearly erred in determining that men usually possessed more of the qualifications for commission selling (even though, as the EEOC argues, Sears has not identified in this data whether the promotions have come from noncommission sales). The EEOC's criticisms go to the hiring figures, and even if they are applied to the promotion data, we do not believe they undermine the essential validity of those figures. We conclude that the judge did not err in inferring from these performance figures that men usually possessed more of the qualifications for commission sales jobs than did women.
109 The district court's conclusions regarding the promotion claim also incorporated its decisions regarding the absence of individual victim testimony and Sears' affirmative action efforts. 41 We have previously determined that these considerations were justified in the context of the claim of hiring discrimination, and see no reason not to apply them to the promotion context as well. Indeed, we think that the absence of individual victim testimony is even more difficult to understand in the promotion context. 110 Although there may have been some reliability problems in Sears' interest adjustments, the question is not whether the interest adjustments alone totally eliminate the promotion disparities. The question is rather whether the court clearly erred in determining that interest adjustments along with other considerations such as differences in qualifications, the lack of individual victim testimony, and Sears' affirmative action efforts negate any inferences of intentional discrimination against women in promotions to commission selling. We conclude that the district court did not clearly err.