the program suppliers critical witnesses a review was pretty deferential with regard to the determination so that they're bound up in these complicated judgments about how best to model which regression  serious discover violation we also have during live testimony resurface indication of the extent of the witnesses work by the witness going in question and that included not just the number of regression models that we now know the had produced all responsive documents which is what did you ask for all i mean how is your argument here you think you asked for this specific information and somehow someone had an improper or unlawful a sufficient response to discovery or you ask th and to me at least this is relevant so if you do anything to enforce your discovery requests not when they responded to us that there were no additional to responsive documents i thought their responses at your request was overbroad so feels to do so and responds that it has produced on response and documents the critical point about this in the way the judge just discovered regulations were you have to produce not just darkness underlin those documents that turned out to be critical because the regression analysis was what the judges relied on for the purposes of determining the show here's and here's here's here's here's the consequences of this not only was the serious discovery violation the witness on the stand was question in any number of ways about whether or not about whether or not he'd performed additional regressions he said no in some cases he can recall they ask well you say it's really important to know how many regression you just made a powerful argument for why it's important no how many how many rounds they went before they got the answer they're testifying about so i'm curious as to why you would have asked that but here again one the one in the risk responding party says we have no addition we have no additional responsive documents that's where stops and we have no reason we take it on good faith that the parties have no additional additional  asking just to confirm if this is really that important at the hearing you could have asked me did you do that you didn't know that the parties had already responded that there were no additional documents when when two parties says there are no response here discovery they just said it was over br that there are no more documents so we don't pursue that discovery this is very different this is a situation where the parties responded that they had no documents and had no responsive documents is very different from we are just the objection to the years both employed service evidence say the way they treated precedent is usually when we talk about an agency changing position something having to explain itself the changes with respect to some legal issues here the change there's a change in the sense that there were prior determinations in which the agency emphasized surveys a supposed to regressions here they use the regression as a starting point and then they look to surveys that seems different and to the extent every every case is different they've explained they've given reasons to think why this regression analysis is better than ones that were tendered in the prior cases doesn't on so they're quite express about what they're doing they do talk about you know in the past you've done more of that but here's the problem with surveys here and here's the much improved  question there are a couple a couple points in that the gist of that determination was actually very inconsistent they said in the same one hundred determination that they were treating the crawford regression analysis in the analysis has massive problems so even if we disagreed on whether or not one was better than the other in this record based on the new revelation that we have six hundred regrets six hundred regression mode tar on behalf of the copyright royalty jokers in the library of congress the challenge for the copyright royalty judges in this proceeding is to decide how to allocate fees the government has collected to copyright owners that provide content to cable systems now the problem is that the fees are set by statute rather than the market and the amount of fees don't generally very based on the type of programming so on this record there were multiple different method ogies and the copyright royalty judges used the methodology offered by dr gregory crawford as a starting point in his analysis that choice was immensely reasonable given systems decide which broadcast distant broadcast systems to retransmit to their subscribers so they have a menu before them and they have to pick which particular programs to retransmit now crawford sought to establish a statistical relationship between the amount of royalties that a cable system paid and the value of that particular programming so the regression he looked at improved on prior regression analyses in multiple ways as thoroughly discussed by the in the final work the biggest improvement is that healing needed simple thing in collecting his data so he had the entire universe of programming for the four years at issue here retransmitted by the form three cable systems that's a difference in measurement between using an abacus and a calculator that's that's the principle advanced that he also looked at every year instead of a sample year and he also used what's called a fixed effects analysis which looked at differences in particular cable systems in something called the subscriber group which was made possible by a congressional amendment congressional enactment in two thousand and ten that a lot  no means perfect there is as this court recognized in its two thousand and five settling devotional claim is decision necessary in precision because we have this mismatch between trying to replicate a relative marketplace on the one hand and a compulsory licensing scheme on the judges judges laying this out but if they did you can point me to it so as i get it what proffered did was correlated the number of minutes of viewership and the amount of royalties paid for each up minutes broadcast with total minutes about all the categories but the difficulty here is that there are many many potential confounding variables that get in the way and so what crawford did in his regression analysis and this starts at j nine thousand one hundred where he sort of lays it out in his appendix and the different control factors he used including for example whether a particular cable system has  material made available to viewers as opposed to viewership numbers but you know from sells because i'm on this record and elsewhere it's pretty easily demonstrated that viewership isn't going up or number and so he's looking at the types of programming that particular cable systems at the subtribe or group level pulled and then retransmit into their individuals and so from that determination you can add up the total number of minutes in for example the sports category or the program suppliers category or the anyone that you can determine value by looking at differentiated program so for example if there are two sorts of sitcoms from the ninety's that are reruns that are being transmitted to this a driver might not care whether it's  one particular sitcom or the other but if there's some sort of meta programming that you really can't find anywhere else and that's unique and then the subscription cable system picks up that particular program then retransmits it and that has a particular type of value so then proffered in the particulars of the regression are laid out it jane one hundred and thirteen years it will take all this information and come to a permanent or marginal basis for the value for each particular type of those six different categories that issue here and then he was able to look at the variety of other different factors but not the payments didn't matter at that point because what they were choosing to transmit the minister yesterday had to look at the total number of units and you pay for a particular what's called a distancing signal  and i thought that's very perplexing because royalties is not a market it's a statutory formula it is for bikes in that it can then and then my law clerk explained to me that the royalty formula the way that works is she just say it's a percentage of revenue you opt into this program and then you pay whatever one percent of your revenue or whatever yes your  vendor and so then i thought well ok what you're really saying is you're correlating on the margin hours to revenue so if you know other things equal you from one year to the next you add an increment of sports programming and then it turns out your revenue goes up by a certain amount controlling for everything else that seems like it's a rough measure of marginal value and so my question is a am i understanding all the fact correctly and be defying yeah why don't you just express the regression as correlating marginal royalties paid by the cable stations are directly proportional to their revenues yes that's correct the current three systems pay i think it's a little more than one percent or something like that so you're correct that it's available programming and these are going out on stations that have been bundled with lots of different programs of what proper had to do was look at not only the station being retransmitted but then also factoring that amount of minutes on that station in the different categories and so i think taking over control variables aside the general purpose is if you're retransmitting more of a certain type of category that category has more val infers that if you're putting out a particular program in a program it's for the purpose of attracting or retrain for a changing subscriber base of so that that is related if i could address the things of that sort but the critical point on this particular record is that program suppliers could have developed it but chose not to were did not and nd crawford's comprehensive complicated whatever word you want to use describe property it's also one hundred percent trans peer and so his entire record by his her analysis which starts on i think in sixty six dollars a joint appendix is transparent for everyone to see for everyone to six hundred times in those results all over the place it looks like the one you wait you didn't didn't didn't tell you got the one you like and then you put it all up with a nice analysis and make it look make me look good but if we knew that you'd actually done a bunch of other six hundred of these it took you six hundred times to get the one that looks good for purposes of this hearing that would have really affected your credibility the look good as i think were the are you really fails if there is something wrong in that particular regression model the parties can point that out and i would just focus on how the program suppliers even in this appeal is only focusing on two particular aspects of crawford's regression model st the allegation that it can't be replicated that's plainly incorrect is true but the theory would be he ran this things six hundred dollars times which is way over the top what you do to prepare regression analysis in a context like this and those results were all over the place as it's before the coverage really three judges in this case and this court now those six hundred purported regression preliminary analyses are before the work from or not because at least there was another party move to strike this testimony a new position whatsoever on whether that discovery should have been turned over or not all i'm merely saying here is that nothing precluded what they had earlier before the hearing when someone said let's rip we we can't have this witness and because we don't they haven't given us and now we have a record where we have proffered information for all to see in print criticised and their criticisms are you know respectfully minor is substantial and now there is this argument that there were but they simply could have developed that in the record before this court just as they did in the satellite allocation proceedings that have now been settled i see my time has r stations that became system carries and a number of those types of states of a combination those are the elements that drive the road to planes there is no combination on any station or a combination of station the combination of or because of programming but the the greater the revenue for the cable company the greater its royalty payment that's correct but might still as a just a very rough wave thinking about kind of how to approximate a market where there is no market what's wrong conceptually with saying we'll look at how you change the programming on the margin and how royalties which correlate to revenue increase so i think that's a that's a very important question but then we don't have  what happens when you look at those systems that actually that she used to pay back more than the minimum payment and what that behavior is like with respect to