Opinion ID: 163083
Heading Depth: 1
Heading Rank: 6

Heading: CWC's ARGUMENTS AND REBUTTAL EVIDENCE

Text: 82 The district court, inter alia, concluded that the 1990 Study, the 1995 Study, the NERA Study, and the other disparity studies upon which Denver relied were significantly flawed because they measured discrimination in the overall Denver MSA construction industry, not discrimination by the City itself. See id. at 1070 (The question, [Denver says], is what happens in the market without [the ordinances]. That may be consistent with scientific methodology but it does not square with the applicable law.). The court may have believed that marketplace data is irrelevant because such data would not answer question four: Would Denver's use of those discriminating firms without requiring them to give work to certified MBEs and WBEs in the required percentages on each project make Denver guilty of prohibited discrimination? Id. at 1066. The district court's conclusion, however, is directly contrary to our holding in Adarand VII that evidence of both public and private discrimination in the construction industry is relevant. See 228 F.3d at 1166-67. 83 Consistent with the district court's conclusion, CWC continues to argue, as it did in Concrete Works II, that marketplace data is irrelevant because the ordinances can be justified only by evidence of discrimination by the City itself or by prime contractors while working on City projects. See supra n. 12. CWC's argument is based on language used in Croson but taken out of context by CWC and by memorandum orders issued by two federal district courts. See Croson, 488 U.S. at 502, 109 S.Ct. 706; Associated Util. Contractors of Md., Inc. v. Mayor & City Council, 83 F.Supp.2d 613, 619 (D.Md. 2000); Webster v. Fulton County, 51 F.Supp.2d 1354, 1368-69 (N.D.Ga.1999), summarily aff'd, 218 F.3d 1267 (11th Cir.2000). In Croson, the Court criticized Richmond's data, in part, because it did not show what percentage of total city construction dollars minority firms now receive as subcontractors on prime contracts let by the city. Croson, 488 U.S. at 502, 109 S.Ct. 706. Relying on this language, CWC argues that the City cannot meet its burden of demonstrating strong evidence of discrimination unless it demonstrates that it directly participates in discrimination or indirectly participates by utilizing contractors who discriminate on City projects. It further argues that evidence of marketplace discrimination is irrelevant to this determination and can never assist the City in meeting its burden. 84 CWC's argument is not supported by the language it cites from Croson when that language is read in context. In the Croson majority opinion, the Court noted, by example, some of the flaws in the evidence presented by the city of Richmond. In addition to criticizing the city because it did not present data showing the percentage of city dollars received by minority subcontractors on city construction projects, the Court also faulted Richmond's evidence that MBE membership in local contractors' associations was extremely low. See id. at 503, 109 S.Ct. 706. The Court's criticism, however, was not that the evidence of low participation in such associations was irrelevant. Instead, the Court criticized the information because it did not consider nondiscriminatory explanations for the low level of membership. See id. Importantly, the Court specifically concluded that if Richmond had linked the evidence showing low MBE participation in trade associations to the number of local MBEs eligible for membership, any resulting statistical disparity, if large enough, could support an inference of discriminatory exclusion. See id. If such an inference could be drawn, the city would have a compelling interest in preventing its tax dollars from assisting these organizations in maintaining a racially segregated construction market. Id. Thus, contrary to CWC's mischaracterization of the Court's statements in Croson, the Court actually concluded that evidence of marketplace discrimination could be used by a municipality to meet its burden of producing strong evidence but that Richmond failed to meet its burden because, inter alia, it failed to offer any such evidence. 85 The conclusion reached by the majority in Croson that marketplace data is relevant in equal protection challenges to affirmative action programs is consistent with the approach later taken by the Court in Shaw v. Hunt. In Shaw, a majority of the Court relied on the majority opinion in Croson for the broad proposition that a governmental entity's interest in remedying the effects of past or present racial discrimination may in the proper case justify a government's use of racial distinctions. 517 U.S. at 909, 116 S.Ct. 1894. The Shaw Court did not adopt any requirement that only discrimination by the governmental entity, either directly or by utilizing firms engaged in discrimination on projects funded by the entity, was remediable. The Court, however, did set out two conditions which must be met for the governmental entity to show a compelling interest. First, the discrimination must be identified discrimination. Id. at 910, 116 S.Ct. 1894 (quotation omitted). The City can satisfy this condition by identifying the discrimination, ` public or private, with some specificity.' Id. (quoting Croson, 488 U.S. at 504, 109 S.Ct. 706) (emphasis added). The governmental entity must also have a strong basis in evidence to conclude that remedial action was necessary. Id. (quotation omitted). Thus, Shaw specifically stated that evidence of either public or private discrimination could be used to satisfy the municipality's burden of producing strong evidence. 86 CWC's argument that the marketplace data is irrelevant is also inconsistent with binding precedent in this circuit. In Adarand VII, we specifically concluded that evidence of marketplace discrimination can be used to support a compelling interest in remedying past or present discrimination through the use of affirmative action legislation. See 228 F.3d at 1166-67 ([W]e may consider public and private discrimination not only in the specific area of government procurement contracts but also in the construction industry generally; thus any findings Congress has made as to the entire construction industry are relevant.  (emphasis added)). Further, in this very case we rejected the argument CWC reasserts here that marketplace data is irrelevant and remanded the case to the district court to determine whether Denver could link its public spending to the Denver MSA evidence of industry-wide discrimination. Concrete Works II, 36 F.3d at 1529. We clearly stated that evidence explaining the Denver government's role in contributing to the underutilization of MBEs and WBEs in the private construction market in the Denver MSA  was relevant to Denver's burden of producing strong evidence. Id. at 1530 (emphasis added). 87 Consistent with our mandate in Concrete Works II, the City attempted to show at trial that it indirectly contributed to private discrimination by awarding public contracts to firms that in turn discriminated against MBE and/or WBE subcontractors in other private portions of their business. Id. The City can demonstrate that it is a `passive participant' in a system of racial exclusion practiced by elements of the local construction industry by compiling evidence of marketplace discrimination and then linking its spending practices to the private discrimination. Croson, 488 U.S. at 492, 109 S.Ct. 706 (O'Connor, J., joined by Rehnquist, C.J. and White, J.). Therefore, evidence of marketplace discrimination is not only relevant but, in this case, it is essential to the City's claim that it is an indirect participant in private discrimination. Consequently, we again reject CWC's argument and conclude that the district court's determination that the marketplace data was irrelevant was a legal error that significantly affected the court's analysis of Denver's evidence. 88 At trial, the City presented testimony from M/WBEs doing business in the Denver MSA to support its assertion that it is a `passive participant' in a system of racial exclusion practiced by elements of the local construction industry because it contracts with firms which discriminate against women and minorities. Id. At least eight M/WBEs testified that general contractors who use them on City construction projects refuse to use them on private projects. The witnesses specifically named the contractors who engaged in this practice. An employee of the MOCC identified eighteen of those contractors as firms that have performed City contracts. Based on this testimony, the district court found that, 89 The City does not want to pay tax dollars to support firms that discriminate against other firms because of their race, ethnicity and gender. Yet, the anecdotal evidence shows that Denver has repeatedly and knowingly done just that. During the taking of testimony about the experiences of minority and woman-owned firms in dealing with other contractors on projects not involving Denver, the City's lawyers carefully demonstrated that those same contractors often do business with the City. 90 Concrete Works III, 86 F.Supp.2d at 1075-76 (discussing narrow tailoring). CWC does not challenge this finding. The anecdotal evidence supporting the district court's finding would not be sufficient on its own to support Denver's burden of demonstrating a strong basis in evidence for its conclusion that remedial action was necessary. However, this evidence links Denver's spending to private discrimination. See Concrete Works II, 36 F.3d at 1529. Consequently, we conclude that the anecdotal evidence and the district court's factual finding based on that evidence amply support Denver's position that it indirectly contributes to private discrimination in the Denver MSA construction industry. See Croson, 488 U.S. at 492, 109 S.Ct. 706 (O'Connor, J., joined by Rehnquist, C.J. and White, J.) (It is beyond dispute that any public entity, state or federal, has a compelling interest in assuring that public dollars, drawn from the tax contributions of all citizens, do not serve to finance the evil of private prejudice.). 91 CWC's argument that the lending discrimination studies and business formation studies presented by Denver are irrelevant is also foreclosed by circuit precedent. In Adarand VII, we concluded that evidence of discriminatory barriers to the formation of businesses by minorities and women and fair competition between M/WBEs and majority-owned construction firms shows a strong link between a government's disbursements of public funds for construction contracts and the channeling of those funds due to private discrimination. 228 F.3d at 1167-68. Evidence that private discrimination results in barriers to business formation is relevant because it demonstrates that M/WBEs are precluded at the outset from competing for public construction contracts. See id. at 1168. Evidence of barriers to fair competition is also relevant because it again demonstrates that existing M/WBEs are precluded from competing for public contracts. See id. Thus, like the studies measuring disparities in the utilization of M/WBEs in the Denver MSA construction industry, studies showing that discriminatory barriers to business formation exist in the Denver construction industry are relevant to the City's showing that it indirectly participates in industry discrimination. 13 92 The City presented evidence of lending discrimination to support its position that M/WBEs in the Denver MSA construction industry face discriminatory barriers to business formation. Denver introduced a disparity study prepared in 1996 and sponsored by the Denver Community Reinvestment Alliance, Colorado Capital Initiatives, and the City. The study's stated purpose was to determine what barriers, if any, small businesses face when seeking credit from financial institutions, and whether there is any difference in treatment among small business owners seeking credit based on their ethnicity. The study ultimately concluded that despite the fact that loan applicants of three different racial/ethnic backgrounds in this sample were not appreciably different as business-people, they were ultimately treated differently by the lenders on the crucial issue of loan approval or denial. In Adarand VII, this court concluded that this very study, among other evidence, strongly support[ed] an initial showing of discrimination in lending. 228 F.3d at 1170; see also id. at 1170 n. 13 (Lending discrimination alone of course does not justify action in the construction market. However, the persistence of such discrimination ... supports the assertion that the formation, as well as utilization, of minority-owned construction enterprises has been impeded. (citation omitted)). The City also introduced anecdotal evidence of lending discrimination in the Denver construction industry. See Concrete Works III, 86 F.Supp.2d at 1072-73. 93 CWC did not present any evidence that undermines the reliability of the lending discrimination evidence but simply repeats the argument, foreclosed by circuit precedent, that it is irrelevant. The district court criticized the evidence because it failed to determine whether the discrimination resulted from discriminatory attitudes or from the neutral application of banking regulations. See Concrete Works III, 86 F.Supp.2d at 1072. We have already concluded, however, that discriminatory motive can be inferred from the results shown in disparity studies. Because the district court's criticism is not directed at the methodology used or conclusions drawn in the lending discrimination study, it does not undermine the study's reliability as an indicator that the City is passively participating in marketplace discrimination. Indeed, in Adarand VII we took judicial notice of the obvious causal connection between access to capital and ability to implement public works construction projects. 228 F.3d at 1170. The district court's other criticisms are erroneous in light of this court's holding that the study support[s] an initial showing of discrimination in lending. Id. 94 Denver also introduced evidence of discriminatory barriers to competition faced by M/WBEs in the form of business formation studies that were included in the BBC and NERA Studies. The 1990 Study and the 1995 Study both showed that all minority groups in the Denver MSA formed their own construction firms at rates lower than the total population but that women formed construction firms at higher rates. The 1997 NERA Study employed a more sophisticated analysis to examine self-employment rates and controlled for gender, marital status, education, availability of capital, and personal/family variables. As discussed, supra, the NERA Study concluded that African Americans, Hispanics, and Native Americans working in the construction industry have lower rates of self-employment than similarly situated Whites. Asian Americans had higher rates. The NERA Study did not calculate business-formation rates for women working in the construction industry. The NERA Study also concluded that minority and female business owners in the construction industry, with the exception of Asian-American owners, have lower earnings than white male owners. This conclusion was reached after controlling for education, age, marital status, and disabilities. 95 Although CWC asserts that the district court correctly concluded that the business formation studies cannot be used to justify the ordinances, the court's conclusion conflicts with our holding in Adarand VII. See 228 F.3d at 1167-68. [T]he existence of evidence indicating that the number of [MBEs] would be significantly (but unquantifiably) higher but for such barriers is nevertheless relevant to the assessment of whether a disparity is sufficiently significant to give rise to an inference of discriminatory exclusion. Id. at 1174. Further, the district court's conclusion that no inference of discrimination can be drawn from the studies because such a conclusion would require the problematic assumption of a direct relationship between the rate of formation of new businesses and the number of persons working as employees in the industry erroneously ignores the fact that the NERA Study controlled for variables like education and level of experience. Concrete Works III, 86 F.Supp.2d at 1066. 96 In its written closing argument, CWC asserted that the business formation study included in the 1995 Study is flawed because it did not control for variables such as the type of educational degree received and types of work experience. 14 Conceding that the NERA Study was better, CWC nevertheless argues that it too is flawed because it did not control for, inter alia, quality of education or culture. The district court faulted the 1995 Study because it did not control for marital status, veteran status, availability of other sources of income and hours worked during the previous year. Concrete Works III, 86 F.Supp.2d at 1057-58. The court also noted that the NERA Study failed to control for prior business experience, religion, cultural history and whether parents were self-employed. Id. at 1060. 97 Even assuming that it is possible to adequately measure variables like quality of education and culture, neither the district court nor CWC provided any explanation of how the failure to control for any of these variables undermines the reliability of the business formation studies. While the report prepared by CWC's expert, George LaNoue, indicated that cultural differences or immigration status may affect business-formation rates, LaNoue did not explain what he meant by the term cultural differences, did not conduct a study that controlled for these variables and, more importantly, did not testify that controlling for the variables would eliminate the disparities. [G]eneral criticism of disparity studies, as opposed to particular evidence undermining the reliability of the particular disparity studies ... is of little persuasive value. Adarand VII, 228 F.3d at 1173 n. 14. 98 In sum, the district court erred when it refused to consider or give sufficient weight to the lending discrimination study, the business formation studies, and the studies measuring marketplace discrimination. That evidence was legally relevant to the City's burden of demonstrating a strong basis in evidence to support its conclusion that remedial legislation was necessary. See id. at 1167-68. CWC's generalized denouncements of the studies fall far short of the credible, particularized evidence necessary to rebut the City's evidence. Id. at 1175.
99 Beyond its conclusion that Denver's disparity studies were largely or wholly irrelevant because they measured marketplace discrimination and not discrimination by the City itself or by contractors working on City projects, the district court identified numerous perceived flaws in the studies. Acknowledging that disparity studies may, when properly conducted, constitute persuasive evidence in equal protection cases, the district court relied on language in Croson 's plurality opinion to set the standards those studies must meet. According to the district court, 100 the probative force of statistical disparity studies depends upon whether the data used provide meaningful measurements of the number of minority firms qualified and willing and able to perform a particular service as well as the number actually used in public contracting, directly or indirectly. 101 Concrete Works III, 86 F.Supp.2d at 1065 (quoting Croson, 488 U.S. at 501, 109 S.Ct. 706). As we have already concluded, disparity studies that measure the use of M/WBEs in the relevant marketplace, in addition to those that measure the direct or indirect use of M/WBEs in public contracting, are also probative. The court, however, also discredited the conclusions reached in Denver's disparity studies either because those studies failed to address the six questions developed by the court or because they failed to control for certain variables. 102
103 In Concrete Works II, this court reversed the grant of summary judgment, in part, because it was unclear from the record whether the 1990 Study overstated the availability of M/WBEs in the Denver construction market. We noted that a disparity index calculated on the basis of the absolute number of MBEs in the local market may show greater underutilization than does data that takes into consideration the size of MBEs and WBEs. Concrete Works II, 36 F.3d at 1528. At trial, both parties had the opportunity to present evidence on the effect firm size and capacity had on the disparities shown in Denver's studies. 104 CWC challenges Denver's disparity studies as unreliable because the disparities shown in the studies may be attributable to firm size and experience rather than discrimination. CWC's argument is based on a report submitted by its expert, George LaNoue, which contains an extensive discussion of firm size and experience. LaNoue first asserts that M/WBEs are generally smaller and less experienced than majority firms. He then contends that smaller and less experienced firms are less qualified and less able to undertake City construction projects. He concludes that because Denver's disparity studies do not control for firm size and experience, the M/WBE availability figures used in those studies are inflated. Thus, CWC argues the disparity indices calculated in the studies are unreliable because they are not based on the availability of only those firms qualified, willing, and able to work on City projects. 105 The district court noted that Denver's disparity studies contain no analysis of the actual qualifications and capacities of the MBEs and WBEs in the Denver MSA, but instead assume that, at any given time, all M/WBEs are available to perform each contract. Concrete Works III, 86 F.Supp.2d at 1065-66. The court criticized Denver's assumption as implausible and concluded that [a]ggregating all of the MBEs and WBEs in estimating availability without regard for the size of the businesses ... is a serious flaw in the methodology and impairs the value of the results. Id. at 1066, 1068. 106 Denver acknowledges that M/WBEs are generally smaller than majority-owned firms in terms of revenues and number of employees and are slightly less experienced. Additionally, in his written report, another CWC expert, John Lunn, asserts that the only conclusion that can be drawn from Denver's studies is that minority-and women-owned firms are smaller and less experienced than the average of all construction firms in the Denver construction market. Although CWC did not conduct its own marketplace disparity study that controlled for firm size and experience, its argument that the disparity studies are unreliable because they fail to control for size and experience is deserving of consideration in light of the uncontroverted evidence that M/WBEs are generally smaller and less experienced than majority firms. 107 Denver counters, however, that a firm's size has little effect on its qualifications or its ability to provide construction services and that M/WBEs, like all construction firms, can perform most services either by hiring additional employees or by employing subcontractors. At trial, Denver introduced evidence that the median number of employees of all construction firms in the Denver MSA is three and presented testimony that even firms with few permanent employees can perform large, public contracts by hiring additional employees or subcontractors and renting equipment. Additionally, the district court found that most firms have few full-time permanent employees and must grow or shrink their performance capacity according to the volume of business they are doing. Id. at 1064. 108 CWC responds that elasticity itself is relative to size and experience; M/WBEs are less capable of expanding because they are smaller and less experienced. The district court found that a firm's ability to expand depends upon [its] access to increased resources, including workers with the needed skills, equipment, material and operating capital. Id. The court did not find that a firm's size affected its ability to obtain these resources but concluded that the firm's access to information, its reputation in the community and the skills of its managers all had an effect. Id. Thus there is no finding by the district court that smaller, less experienced firms are less able to expand. 109 Even if we assume that M/WBEs are less able to expand because of their smaller size and more limited experience, CWC does not respond to Denver's argument and the evidence it presented showing that experience and size are not race-and gender-neutral variables and that M/WBE construction firms are generally smaller and less experienced because of industry discrimination. The lending discrimination and business formation studies both strongly support Denver's argument that M/WBEs are smaller and less experienced because of marketplace and industry discrimination. In addition, Denver's expert David Evans testified that discrimination by banks or bonding companies would reduce a firm's revenue and the number of employees it could hire. Robin Hackett, the owner of a WBE, testified that she has difficulty finding and retaining white male employees because they are ridiculed by others in the industry for working at a WBE. Hackett also testified that she is excluded from participating in a labor pool of non-union employees shared by other local firms. Those firms sharing this labor pool are able to bid a bigger volume of work and not worry about manning the projects. Even Lunn, CWC's expert, admitted during cross-examination that the size and revenue differences between M/WBEs and majority-owned firms could be due in part to the presence of discrimination. 110 Denver also argues that CWC's argument lacks merit because the 1990 Study, the DGS Study, the 1995 Study, and the NERA Study all controlled for size and the 1995 Study controlled for experience. It asserts that the 1990 Study measured revenues per employee for construction M/WBEs and concluded that the resulting disparities, suggest[] that even among firms of the same employment size, industry utilization of MBEs and WBEs was lower than that of non-minority male owned firms. Similarly, the 1995 Study controlled for size, calculating, inter alia, disparity indices for firms with no paid employees which presumably are the same size. CWC responds by simply making the bald allegation that the 1995 Study's control for size is a pretense and that the control is meaningless. This glib response is wholly inadequate to rebut Denver's argument and the conclusions drawn in the disparity studies. 111 Based on the uncontroverted evidence presented at trial, we conclude that the district court did not give sufficient weight to Denver's disparity studies because of its erroneous conclusion that the studies failed to adequately control for size and experience. Denver is permitted to make assumptions about capacity and qualification of M/WBEs to perform construction services if it can support those assumptions. The assumptions made in this case are consistent with the evidence presented at trial and support the City's position that a firm's size does not affect its qualifications, willingness, or ability to perform construction services and that the smaller size and lesser experience of M/WBEs are, themselves, the result of industry discrimination. Further, CWC did not conduct its own disparity study using marketplace data and thus did not demonstrate that the disparities shown in Denver's studies will decrease or disappear if the studies controlled for size and experience to CWC's satisfaction. Consequently, CWC's rebuttal evidence is insufficient to meet its burden of discrediting Denver's disparity studies on the issue of size and experience. 112
113 The district court also faulted Denver's disparity studies because they do not control for firm specialization. See Concrete Works III, 86 F.Supp.2d at 1068 (Aggregating all of the MBEs and WBEs in estimating availability without regard for... the particular services or type of work in which they specialize is a serious flaw in the methodology and impairs the value of the results.). The court's criticism would be appropriate only if there was evidence that M/WBEs are more likely to specialize in certain construction fields. CWC has failed to marshal any such evidence. Its appellate brief contains only the conclusory statement that M/WBEs tend to be... congregated in certain construction specialities. This assertion is unsupported by any citation to the record and CWC does not direct this court to any corroborative study or other evidence. Further, there is no identified evidence showing that certain construction specializations require skills less likely to be possessed by M/WBEs. In fact, CWC's statement appears to contradict the testimony of the City's expert, Dr. Evans, that the data he reviewed showed that MBEs were represented widely across the different [construction] specializations. 15 114 In his expert report, Lunn criticizes Denver's disparity studies because they aggregate construction firms without regard to specialization or size. Although Lunn's report contains an example that illustrates his criticism, he conceded during cross-examination that the example was purely hypothetical and was not based on any data from the Denver MSA construction industry even though he had access to that data. Lunn also testified that he had not done any investigation into whether aggregation bias caused the disparities shown in Denver's studies. 115 Not only does CWC offer no support for its position that M/WBEs are clustered in certain construction specialities, but it has failed to demonstrate that the disparities shown in Denver's studies are eliminated when there is control for firm specialization. In contrast, the NERA Study, which controlled for SIC-code subspecialty yet still showed disparities, provides support for Denver's argument that firm specialization does not explain the disparities. 116
117 In his report, CWC's expert, LaNoue, contends that the M/WBE availability data used in Denver's disparity studies is unreliable because it is not a measure of only those firms actually bidding on City construction projects. LaNoue asserts that, All other measures than bidding are merely proxies for availability.... [U]nless a firm makes a bid, it is not actually available for public contracting. LaNoue's position appears to constitute an implicit conclusion that only studies which equate M/WBE availability with actual bids on a contract-by-contract basis can be used to support Denver's burden. To calculate availability at the level of certainty urged by CWC, Denver would be compelled to survey each MBE, WBE, and majority firm each time bids are sought for a construction project to determine which firms actually bid on the project, either as a prime contractor or as a subcontractor. While this approach may provide an accurate count of available firms, it says nothing about whether those firms are qualified. If availability is calculated on an individual basis using LaNoue's approach, it is possible that unqualified firms would be included in the availability figure simply because they bid on a particular project. Conversely, qualified firms would not be included if, for any reason, they chose not to bid on the project. Thus, LaNoue's approach itself illustrates why disparity studies must make assumptions about availability as long as the same assumptions can be made for all firms. 16 118 CWC does not identify any evidence showing that M/WBEs bid on City projects at a different rate than non-M/WBE firms. Thus, it has not demonstrated that differences in bidding practices may explain the disparities found in Denver's studies. Additionally, we do not read Croson to require disparity studies that measure whether construction firms are able to perform a particular contract. The studies must only determine whether the firms are capable of undertak[ing] prime or subcontracting work in public construction projects. Croson, 488 U.S. at 502, 109 S.Ct. 706.
119 John Lunn, CWC's expert, criticized Denver's studies because they used stock data and not flow data. Studies utilizing stock data provide information on an industry at a discrete point in time. Flow data show changes that occur in the industry over time. Lunn testified that substantial changes in a market situation and substantial changes in flows do not show up in studies utilizing the stock concept. He then speculated that a study utilizing flow data might show no disparities even though an analysis of stock data may indicate that discrimination is present. Lunn, however, illustrated his point by using a hypothetical involving law school admissions: he did not test his hypothesis on actual flow data from the Denver construction market and he did not testify that any substantial change had occurred in the Denver construction industry which would make a study using flow data more reliable than the disparity studies conducted by BBC and NERA. 120 Denver's individual studies examined the Denver construction industry from the 1970s through 1997. Each study showed disparities between M/WBEs and non-M/WBEs. Thus, considered in the aggregate the disparity studies themselves undermine CWC's unsupported assertion that the use of flow data would explain the disparities.
121 CWC argues that Denver cannot demonstrate a compelling interest because it overutilizes M/WBEs on City construction projects. This argument is an extension of CWC's argument that Denver can justify the ordinances only by presenting evidence of discrimination by the City itself or by contractors while working on City projects. Because we have concluded that Denver can satisfy its burden by showing that it is an indirect participant in industry discrimination, CWC's argument relating to the utilization of M/WBEs on City projects goes only to the weight of Denver's evidence. 122 In Concrete Works II, this court concluded that CWC had raised a genuine issue of material fact relating to Denver's reliance on data from City-funded construction projects not subject to the ordinances or the goals program which predated the ordinances. We noted that one set of data reveal[ed] extremely low MBE and WBE utilization. Concrete Works II, 36 F.3d at 1526 (referring to the DGS Study). Similarly, data analyzing the utilization of M/WBEs on bond projects for which goals were never set also showed underutilization of M/WBEs. See id. at 1527 (referring to the 1990 Study). In contrast to this data, however, the record also contained studies showing very strong M/WBE utilization on City projects subject to the goals program. See id. at 1525. Consistent with this court's mandate in Concrete Works II, at trial Denver sought to demonstrate that the utilization data from projects subject to the goals program was tainted by the program and reflect[ed] the intended remedial effect on MBE and WBE utilization. Id. at 1526. 123 In the pretrial order, the parties stipulated that the City's utilization of MBEs 17 exceeded their availability in 18.75 of the 20.75 years measured since 1978. CWC asserts that this indicates there is no underutilization of MBEs by the City and, thus, data on the utilization of MBEs on City projects provides no support for the race-based remedial measures. 18 Denver argues that the non-goals data is the better indicator of past discrimination in public contracting than the data on all City construction projects. 124 At trial, Denver presented evidence that the high utilization of MBEs on City projects subject to the goals program was the result of the program and not the absence of discrimination. Particularly persuasive was Denver's evidence that MBE utilization rates on City construction projects declined significantly after the goals program was amended in 1989. One amendment eliminated the minimum annual goals for MBE participation and replaced them with a system whereby goals were set for each individual project. Another amendment required MBEs seeking certification to demonstrate that they had suffered from past discrimination. The drop in utilization rates can thus be linked to a relaxation of the City's affirmative-action efforts. CWC presented no evidence that otherwise explains the decrease in utilization rates after the amendments became effective. 125 Denver also presented uncontroverted testimony that the goals and non-goals projects were similar. One of Denver's experts testified that the same pool of construction firms worked on both the goals and non-goals projects. Additional evidence indicated that projects that should have been performed by the DPW were diverted to the DGS to avoid the goals program. This demonstrates that there was at least some overlap in the character of the goals projects administered by DPW and the non-goals projects administered by the DGS. Again, CWC presented no evidence that the character of the projects was significantly different or that they involved different pools of prime contractors. Finally, CWC's argument that overutilization of MBEs on City projects rendered MBEs unavailable to work on private projects has little persuasive value in light of the extensive evidence on industry elasticity and Denver's evidence that in 1989, DPW projects accounted for less than 4% of all MBE revenues. We conclude that Denver presented ample evidence to support the conclusion that the evidence showing M/WBE utilization on City projects not subject to the ordinances or the goals programs is the better indicator of discrimination in City contracting. 126 CWC also attacks the non-goals data presented by the City and the conclusions drawn from that data. Because CWC takes the position that the City can only remedy its own discrimination or the discrimination of contractors while working on City projects, it asserts that the non-goals data is the only relevant data presented by Denver. It further argues that the data provides insufficient support for Denver's burden. We have rejected CWC's argument that the marketplace data is irrelevant but agree that the non-goals data is also relevant to Denver's burden. We note, however, that Denver did not rely heavily on the non-goals data at trial but focused primarily on the marketplace studies to support its burden. 127 CWC attempts to rebut the non-goals evidence by noting flaws in some of the disparity indices calculated on non-goals projects in the 1990 Study. CWC first criticizes the portion of the 1990 Study which examined the eight local bond projects undertaken between 1972 and 1976. CWC asserts that the disparity indices calculated for these bond projects are meaningless because they were calculated using 1977 availability figures. This valid criticism undermines the persuasive value of these disparity indices. CWC also challenges the conclusions drawn in the portion of the 1990 Study that analyzed five private housing projects constructed in 1985 with revenue bonds but not subject to the goals program. The flaws CWC notes in this study are legitimate and support its position that the minority utilization figures are inaccurate and, thus, the subject disparity indices in the 1990 Study may not be reliable. 128 CWC also asserts that Denver cannot rely on the Peat Marwick study to support the ordinances because that study found no evidence of discrimination in bonding by the City. Unfortunately, we cannot evaluate the merits of CWC's argument because it has provided no record citation directing us to this study and we were unable to locate it in the appellate record. We also note that while it was discussed in Concrete Works I, it is not referred to in Concrete Works III. After a time-consuming search of the trial transcript, unguided by any record citation, we were able to locate and review only one piece of relevant evidence: the testimony of a witness, Wesley Martin. On cross-examination the following exchange occurred between Martin and CWC's counsel, 129 Q: — you would agree that they did find there that there was no evidence of bonding discrimination in the bonding industry? 130 A: I believe they termed it a little bit differently than that, if I recall from reading that. They said that there was no immediate evidence of discrimination evident there, but I don't think they came to a conclusion that it did not exist. 131 This testimony provides no support for CWC's one-sentence argument that the Peat Marwick study found no evidence of discrimination, either by Denver or by the bonding industry. Because we cannot review the study itself, we cannot determined whether it rebuts Denver's other evidence and we conclude that CWC's argument is waived. 132 The City also relied on the DGS Study, the GAO Report, and the DOT's threat to withdraw federal funding as evidence of City discrimination on non-goals projects. CWC does not rebut or even discuss the DGS Study which showed disparity indices of 0.14 for MBEs and 0.47 for WBEs on 1989 DGS remodeling projects. On 1990 DGS remodeling projects the disparity indices were 0.19 for MBEs and 1.36 for WBEs. With respect to the GAO Report, CWC asserts that the only finding reached in that report was that the DPW's prequalification requirement had a disparate impact on minorities. CWC then summarily argues that this finding does not support the conclusion that the City was discriminating because the DPW's prequalification requirements were race neutral. The GAO Report, however, concluded that the prequalification system was applied in an inconsistent and subjective manner and that it appeared to have a greater adverse effect on small and minority contractors. The Report also concluded that the DPW's advertising procedures for bid proposals and its lack of a procedure to assure that contractors had adequate time to prepare bid proposals had a greater negative effect on smaller and/or less experienced contractors and minority contractors. Denver has already conceded that MBEs are generally smaller and less experienced than majority-owned firms and has introduced evidence indicating that both characteristics are the result of industry discrimination. See § VI.A.2.a, supra. CWC also asserts that Denver's reliance on the GAO Report is misplaced because the Report contained an incorrect minority availability figure of 11%. In the 1990 Study, BBC used census data to estimate MBE availability in 1977 19 at 2%. The GAO Report, however, did not calculate disparity indices or attempt to determine whether minority contractors were being underutilized. As CWC itself concedes, the Report concluded only that Denver's contracting practices and policies adversely affected small contractors and minority contractors more than majority contractors. CWC's criticism does nothing to rebut this finding. As the Report itself indicates, the GAO did not rely on the 11% availability figure to reach its conclusions: 133 Since Denver had not established specific goals for minority contractor participation in federally assisted construction contracts, no measure was available to determine whether the current 5-percent level of participation was reasonable. However, Denver identified about 11 percent of the construction contracting firms in the Denver metropolitan area as being owned by minorities. 134 Although the GAO Report is not particularly compelling because it contains very little analysis, the conclusions drawn in the Report do provide some support for Denver's position that it engaged in discriminatory conduct. CWC's arguments are insufficient to rebut those conclusions. 135 CWC's attempt to discredit Denver's reliance on the DOT's threats to withdraw federal funding is also unpersuasive. Like the GAO Report, the DOT found that Denver's prequalification requirements operated as a barrier to minority firms. In 1979, the DOT threatened to initiate administrative proceedings to terminate federal funding for Stapleton International Airport unless Denver modified its prequalification procedures and developed an affirmative action plan applicable to Stapleton. We again reject CWC's argument that the DOT's findings do not show discriminatory conduct by Denver because the prequalification requirements were race-neutral. CWC also asserts that the DOT's findings are flawed because the DOT erroneously determined that minority contractor utilization was 6% in 1978 and 5% in 1979. CWC again points to the 1990 Study prepared by BBC which calculated MBE availability in 1977 at 2% and argues that, based on the DOT's utilization figures, MBEs were actually overutilized on DOT-funded projects. CWC does not provide any record citation to support its assertion that the DOT determined minority utilization to be 6% in 1978 and 5% in 1979. We have reviewed all the materials in the appellate record relating to the City's dispute with the DOT and could not locate the source of CWC's information or confirm its accuracy. Thus, we are unable to consider CWC's argument. In addition, we question whether an accurate disparity index can be calculated by using availability figures from one study and utilization figures from a second study. There is simply no way to determine whether similar data-collection methods were used in the two studies. 136 In sum, Denver has demonstrated that the utilization of M/WBEs on City projects has been affected by the affirmative action programs that have been in place in one form or another since 1977. Thus, the non-goals data is the better indicator of discrimination in public contracting. CWC's criticisms of the non-goals disparity indices reported in the 1990 Study persuasively rebuts that data. CWC, however, is unable to persuasively rebut the other non-goals evidence, including the DGS Study, the GAO Report, and the DOT's threatened administrative action. We conclude that, on balance, the non-goals data provides some support for Denver's position that its belief that racial and gender discrimination existed in public contracting before the enactment of the ordinances was supported by strong evidence.
137 The 1995 Study contained tables showing that revenues per employee were lower for Hispanic, Asian, Native American, and women-owned construction firms in the Denver MSA than for majority-owned firms. CWC argues that this data on revenues is flawed. In his expert report, CWC expert, LaNoue, asserted that prime contractors report gross revenues differently than subcontractors: a subcontractor counts only those revenues earned by its own employees while a prime contractor also includes amounts it receives but pays out to subcontractors. At trial, LaNoue testified that all the evidence suggests that white, male-owned firms are more likely to be prime contractors. Thus, CWC argues that the revenue comparisons contained in the 1995 Study overstate the revenues of majority-owned firms. 138 CWC's argument rises or falls on the correctness of the statement in LaNoue's expert report that MWBEs, as newer, smaller firms, tend more often to be subcontractors than prime contractors. LaNoue did not identify any study or other evidence upon which he based his statements. In fact, availability data reported in Table 2 of the NERA Study appear to contradict LaNoue's statement. That data shows that both MBEs and WBEs are almost equally likely to be prime contractors as they are to be subcontractors. When an expert opinion is not supported by sufficient facts to validate it in the eyes of the law, or when indisputable record facts contradict or otherwise render the opinion unreasonable, it cannot support a jury's verdict. Brooke Group Ltd. v. Brown & Williamson Tobacco Corp., 509 U.S. 209, 242, 113 S.Ct. 2578, 125 L.Ed.2d 168 (1993). 139 LaNoue's statements, at best, provide tenuous support for CWC's argument. However, Denver does not challenge the district court's finding, apparently based on LaNoue's unsupported testimony, that small minority firms most often work as subcontractors. Concrete Works III, 86 F.Supp.2d at 1070. Thus, we must credit CWC's argument but conclude it is only marginally persuasive and further note that Denver's reliance on the revenue-per-employee data was minimal. 140 To support its related argument that majority-owned firms have greater revenues per employee because they are larger and more experienced, CWC offered the testimony of Craig Long. Long, who has worked in the Denver construction industry for twenty-eight years, testified that larger and/or more experienced firms are more competitive and more profitable per employee because they, inter alia : (1) have more access to cash so they can take advantage of early-payment discounts; (2) can obtain lower prices from suppliers if they have established long-term relationships with them; and (3) can obtain credit at lower rates. We have already concluded, however, that M/WBEs are less able to benefit from the advantages Long describes because of discrimination. 141 CWC also argues that the revenue comparisons are flawed because they do not identify the source of the revenues. CWC takes the position that Denver cannot use evidence of revenue differences to show that it is a passive participant in marketplace discrimination unless it can demonstrate that all revenues were derived from work done in the Denver MSA. It appears uncontested that Denver's studies did not exclude revenues that Denver-based firms earned on projects outside the Denver MSA. See id. (noting that the data used in the NERA Study do not limit revenues to what was earned by work done in Colorado). LaNoue testified that ascertaining the source of firm revenues is important because it identifies the entities or individuals responsible for the discrimination. Given our conclusion, supra, that Denver's other evidence supports the conclusion that it is a passive participant in marketplace discrimination, CWC's criticisms regarding the source of revenues are irrelevant.
142 Faced with the prodigious anecdotal evidence presented by Denver at trial, CWC argues that many of the examples of discrimination are not severe or extreme. The anecdotal evidence, however, included several incidents involving profoundly disturbing behavior on the part of lenders, majority-owned firms, and individual employees. The anecdotal testimony revealed behavior that was not merely sophomoric or insensitive, but which resulted in real economic or physical harm. While CWC also argues that all new or small contractors have difficulty obtaining credit and that treatment the witnesses characterized as discriminatory is experienced by all contractors, Denver's witnesses specifically testified that they believed the incidents they experienced were motivated by race or gender discrimination. They supported those beliefs with testimony that majority-owned firms were not subject to the same requirements imposed on them. 143 There is no merit to CWC's argument that the witnesses' accounts must be verified to provide support for Denver's burden. Anecdotal evidence is nothing more than a witness' narrative of an incident told from the witness' perspective and including the witness' perceptions. In this case, the anecdotal evidence was not subject to rigorous cross-examination. See id. at 1071 (This court limited the number of witnesses to be called for this purpose to avoid duplication under F.R.E. 403, received lay opinions under F.R.E. 701, expansively accepted hearsay and applied a liberal standard of relevance. Cross examination of these witnesses was restricted to avoid unduly prolonging the trial.). Denver was not required to present corroborating evidence and CWC was free to present its own witnesses to either refute the incidents described by Denver's witnesses or to relate their own perceptions on discrimination in the Denver construction industry. 144 After considering Denver's anecdotal evidence, the district court found that the evidence shows that race, ethnicity and gender affect the construction industry and those who work in it and that the egregious mistreatment of minority and women employees had direct financial consequences on construction firms. Id. at 1074, 1073. The court, however, failed to credit any of Denver's anecdotal evidence because it did not answer the six questions posed by the court. See id. at 1074 ([T]aken as a whole, the anecdotal evidence does not answer the six questions considered in the court's evaluation of the statistical evidence.). As we conclude supra, however, the district court's six questions represent a misstatement of the governing law and imposed an erroneously high burden on Denver. Based on the district court's findings regarding Denver's anecdotal evidence and our review of the record, we conclude that the anecdotal evidence provides persuasive, unrebutted support for Denver's initial burden. Cf. Int'l Bhd. of Teamsters v. United States, 431 U.S. 324, 339, 97 S.Ct. 1843, 52 L.Ed.2d 396 (1977) (concluding that anecdotal evidence presented in a pattern or practice discrimination case was persuasive because it brought the cold [statistics] convincingly to life).
145 The record contains extensive evidence supporting Denver's position that it had a strong basis in evidence for concluding that the 1990 Ordinance and the 1998 Ordinance were necessary to remediate discrimination against both MBEs and WBEs. The information available to Denver and upon which the ordinances were predicated, indicated that discrimination was persistent in the local construction industry and that Denver was, at least, an indirect participant in that discrimination. 146 Before enacting the 1990 Ordinance, Denver was informed by the GAO that the DPW was likely not complying with federal affirmative-action requirements. A year after the GAO Report, the DOT threatened to withdraw federal funding for Stapleton International Airport because the DOT took the position that minority contractors were not being utilized at appropriate levels. The City then held public hearings at which extensive testimony was given detailing discrimination against MBEs and WBEs on local construction projects. In 1988, the City solicited written responses to questionnaires and conducted additional hearings on the issue of discrimination in the local construction industry. 147 The City then retained BBC to evaluate the utilization of M/WBEs on local construction projects. BBC ultimately produced the 1990 Study which showed large disparities in the utilization of MBEs and WBEs on construction projects undertaken in the Denver MSA. Further, the 1990 Study indicated that there were disparities in the utilization of M/WBEs on City projects. The anecdotal evidence collected by BBC and summarized in the 1990 Study strongly suggested that there was ongoing discrimination against MBEs and WBEs in the Denver construction industry. 148 Before enacting the 1998 Ordinance, Denver had the opportunity to evaluate the considerable amount of information contained in the 1995 Study. That study used both census data and information obtained from a telephone survey to examine the utilization of MBEs and WBEs in the Denver construction and professional design industries. The census data showed disparities in the utilization of MBEs and the survey data showed disparities in the utilization of MBEs and WBEs. The 1995 Study also showed that Blacks and Hispanics working in the Denver construction industry had lower rates of self-employment than Whites and that women had lower rates of self-employment than men. The disparities in self-employment rates persisted even though the 1995 Study controlled for education and length of work experience. 149 The 1995 Study also contained a summary of the RTD Study and the DHA Study. The RTD Study showed disparities in the utilization of M/WBE prime contractors and subcontractors in the Denver MSA construction industry. The DHA Study found some disparities in the utilization of M/WBEs on DHA projects. In addition, the separate DGS Study showed disparities in the utilization of MBEs and WBEs on DGS construction and remodeling projects. 150 The City then retained NERA to examine whether discrimination limited M/WBE participation in contracting. The NERA Study showed disparities in the utilization of M/WBEs in the Colorado construction industry. It also concluded that African Americans, Hispanics, and Native Americans working in the Denver MSA construction industry have lower self-employment rates than Whites. The NERA Study then used the information on self-employment rates to calculate the potential availability of MBEs if they formed construction businesses at the same rate as Whites. It then compared potential availability to the actual availability of MBEs in the Denver MSA. In the case of African American, Hispanic, and Native American prime contractors and subcontractors, potential availability exceeded actual availability. 151 The NERA Study also examined the earnings of self-employed minorities and women in the construction industry. The study showed disparities in the earnings of self-employed women, African Americans, Hispanics, and Native Americans. Finally, the results of a mail survey conducted by NERA showed that M/WBEs engaged in business activities experience disparate treatment at a higher rate than non-M/WBEs. 152 To meet its initial burden, Denver was not required to unequivocally establish the existence of discrimination nor was it required to negate all evidence of non-discrimination. Concrete Works II, 36 F.3d at 1530. After properly considering all relevant evidence presented by Denver at trial and the district court's undisputed factual findings, we conclude that Denver met its initial burden of producing strong evidence of racial discrimination in the Denver construction industry. Denver has also shown that the gender-based measures were based on reasoned analysis. Moreover, although CWC does not raise the issue, we conclude that Denver had a strong basis in evidence to conclude that action was necessary to remediate discrimination against M/WBEs before it adopted both the 1990 Ordinance and the 1998 Ordinance. See Shaw, 517 U.S. at 910, 116 S.Ct. 1894 ([T]he institution that makes the racial distinction must have had a strong basis in evidence to conclude that remedial action was necessary, before it embarks on an affirmative-action program. (quotation omitted) (emphasis in original)). 153 To rebut Denver's evidence, CWC was required to establish that Denver's evidence did not constitute strong evidence of such discrimination. Concrete Works II, 36 F.3d at 1523. CWC cannot meet its burden of proof through conjecture and unsupported criticisms of Denver's evidence. Rather, it must present credible, particularized evidence. Adarand VII, 228 F.3d at 1175. CWC has not met its burden. Its legal arguments urging us to disregard Denver's non-goals data, marketplace data, lending discrimination study, and business formation study lack merit. Further, its criticisms of the methodologies used in those studies, albeit legitimate in some very limited circumstances, merely nip at the edges of Denver's evidence and are insufficient to undermine the reliability of the studies. CWC hypothesized that the disparities shown in the studies on which Denver relies could be explained by any number of factors other than racial discrimination. However, it did not conduct its own marketplace disparity study controlling for the disputed variables and has presented no other evidence from which this court could conclude that such variables explain the disparities. Although CWC advanced a seemingly meritorious argument that the size and experience of M/WBEs may explain the disparities, Denver's additional evidence persuasively dispelled that argument. 154 [W]hether a strong basis in evidence of past or present discrimination exists, thereby establishing a compelling interest for the municipality to enact a race-conscious ordinance, is a question of law. Concrete Works II, 36 F.3d at 1522. We conclude that Denver has demonstrated that a strong basis in evidence supported its conclusion that remedial action was necessary to remedy racial discrimination in the Denver construction industry. Further, Denver has shown that the gender-based measures in the 1990 and 1998 Ordinances were based on reasoned analysis. Consequently, we conclude that Denver has shown that it has a compelling interest in remedying racial discrimination in the Denver construction industry and that it has an important governmental interest in remedying gender discrimination. CWC has failed to rebut Denver's showing.