please the court just i'm president of the united states i'd like to reserve five minutes for a bottle the district court aired when it elevated power from a gate keeping function to a much more exacting standard dauber a says anything that is not junk science should be admitted in tested by the adversarial process of cross examination and if available a qualified competing expert but that's not what the among probably stick genotyping software but you know just science is you know pejorative and i'm not sure that's exactly how the line works but if we're going to go down that road in those terms you're going to use it is junk science at some point right i mean don't you agree that there are some level at which it should be admitted it has the risk of misleading jurors well well your honor i respectfully disagree that you government's position no no your turn to clarify that is not the government's position the government's position is the software has been proven to basically work down to no d n a it's still requirement that the internal validation study cover to probably as to geno type being in general instruments in particular but starlix has proven generally that it clears that threshold and it's also proven that in this particular case that michigan state police  test at those levels in order for us to reverse the key point i'm trying to make as why why if we focus on it just on the specific facts of this case it just records the opinion could be read very you're correct can be read literally it's also can be read quite broadly because it makes broad statements about the technology and how it's not really evidence at all because it's to come complicated in blinds on all these theories but even if you just focus on the narrow portions specific portion  about this interaction effect arose the michigan state police said well we've done experiments like that we just don't write up the data in the summary of the internal allegations to be because we don't think that was important but what's what's wrong with the  argument wasn't raised in the initial motion or at the initial hearing or in the supplemental briefing after the initial hearing and i think the record will look quite different if it had been because then those issues would have been joined you know the government that issue and doesn't actually think dr crane's criticism is valid so i think the court can say looking at all of the evidence and testing including the low levels of d n a if you look at the developmental validations published studies the work that the michigan state police did and you look at all of it i think the i think you can find an abuse of discretion in the way the court weigh the evidence but i also think that you can say if if a year and a half into the case if you're going to rely on this new argument and dr klein says this is what i need to see in the government shows it and then dr crane says well no actually now i want to see the electra fairgrounds and i want to see the likelihood ratio then yes you should give the government an opportunity to offer that evidence of if that's going to be critical now the problem is though so just again just to make sure i'm getting the point that's really not an abuse of discretion ruling that's a ruling that says the district court jumped the gun they needed to give the government another chance to respond and then we'll decide then we'll look at it is that's what it sounds like the way you just framed well well i think that's part of the problem now the government is asking for for vacating for further evidence because i think this process that we're going through would really just keep going you show me a letter a fairgrounds and he would say well i want to see something else then you show the likelihood ratios and you'd say well let's investigate this likelihood ratio and you can always i any scientific process you can always say well let's look at more data more evidence in the government's view is that doubt does not require the defense expert to be invited into the laboratory for the internal validation study there's a certain point at which you have to say well we need to see this evidence the lab did it in you know in the lab showed that they conducted the experiments that he was concerned about which is corroborated by all of the other evidence that has all the other studies that have been done about t key question to be answered in determining whether a theory or technique is scientific knowledge that will assist to your trier of fact will be whether it can be and has been tested and so i guess the question i have for both of you of both sides as what's the level of generality at which reacted is herman whether it has been tested because obviously stomach's has been rigorously tested generally and there's also plenty of evidence to suggest that it has been tested at lower levels just at other labs is that enough is that legal question what this language means by how much testing is necessary and mike is that is that enough under under this test or do you have to actually boil down to the specific facts of this case when asked whether it has been tested under the specific facts you're right your honor i think it's both i think there is a legal component when you say well what evidence is the district court supposed to look at under the testing factor that is that is a legal question i think the answer to that is it's supposed to look at all of the evidence right so all the evidence before it which in this case includes the developmental validation  the individual laboratory has to conduct internal validation and i think you can look to the standards factor for the swing dance here the standard here the standards and guidelines and there's no distinction between standards and guidelines and in the way that it's used in the daubert framework here are the guidelines you have to follow when you're conducting an internal validation study in there was no dispute in the district court or really until the media filed there or if that the michigan state police fully complied with the internal with the split game guidelines when they conducted their internal validation study and in the immediate for reasons we explained reply brief or just flat out wrong when they say michigan state police were consistent with you know one of the government's witnesses i'd like to respond to a couple of the government's points but before i do so i st want to underscore that this is not a challenge to star mix per se as i think the court has recognized this is a challenge to a case by a particular lab on a particular type of sample of another th a mixtures into it involving a low minor contributor very low and a very tiny amount of d n a there are many subjective decisions as the record just demonstrates along with different steps in the system and these come out upon one another all of these factors influence the up ultimate likelihood that likelihood ratio that's presented to the jury eroding its relevance and its reliability is special factor because you could when it talks about whether a theory or technique can be or has been tested it's obvious d that it can be tested even so they can be dissatisfied and that has been there's also been a lot of testing with respect to the theory or technique if we interpret theory or technique just generally yes duramax now your response i actually think would be that you should interpret dalbir d to suggest that the theory or tech technique means with the specific simple models and weights percentages and i'm wondering if if that's a cryptic correct that seems like no we won't question to me about what dahlberg means when it says it's can be tested or has been tested do you have any cases to and that we're being independent and testing and it needs to be robust and history robust on these types of samples not necessarily precisely because levels it if you're looking at the internal validation for example that michigan state police compl a financial incentive ideally we would have some sort of functional equivalent to say the national highway traffic traffic safety administration completing some sort of objective robust testing perhaps nist could do that it may not be its role but incidentally that was the aim of the pink ass report and what they were trying to encourage to bring together many experts without a vested interest in the outcome just scrutinize certain areas of forensic science and that's what good argument is based on a an assumption that there are very conditions from one lab to another here that affect the reliability of the results is that basically what you're saying that's  validation study is required and the community i grant that so you do have to perform an internal validation study every single time you get the software so that's already a component but if you're going to perform that what what is that review i mean what does downwards say the route you has to be because it's really just the level of review i mean you can have a more general lack of internal validation or you can have a less generalized but you can always make it more government's ability to comply with the last three factors it seems to me the last three factors of very strongly support the use of storms that leaves the st factor ok and i fell occur in alice as it gets close to abuse of discretion in terms of not appreciating the way the last three factors favor the use of storms as to the st factor which clearly is the key point it does seem a little strange they didn't give the government one more chance to get those numbers i mean if that's what the community wants why not give them one more chance and then my the last part of that approach would acknowledge just because storm x may be cleared out there doesn't mean that this kind of the fat factor i think that this within the federal government itself amongst the experts some of between the statisticians between the people who testified in this particular case shows that there are still disagreement about its application specifically are you defense contrast that with dr buckle sense testimony there is disagreement here about whether or not this has been a reliable for these types of samples and that goes to all of the perry be a factor  publication important component the author my understanding to review is so even if the author is biased the scientific community when they decide whether to publish this article is reviewing providing the consider the author and that case it's not going to there's no bright line rule whether or not it an author is involved in a parody of publication in this case almost all one of the publication to involve an author and i think at that point in time you have to start questioning whether they've ever been meaningfully parried i did want to bring up to address your question judge putting the final factors with regard to the error rate and are generally accepted guidelines perhaps generally accepted within the law enforcement community but you can say that the fact that the community can't even great agree as of today as to what those standards should be for both interpretation validation of the software just the lies the ongoing dispute and the lack of general acceptance for this type of technology as it applies to precisely base types of samples first and foremost the lack of independence if you look at the labs that were performing this again they have a mission oriented objective their law enforcement labs the study again was a fembot old and authored by the creators of star mix for the sole purpose of defending a product in which they take a lot of understandable personal pride you have a lack of independence and that's a very important very important consideration or if you review the article closely to it shows that there is a problem due to human error in the underestimation of the number of contributors and the testimony from both adams and crane demonstrates that an underestimation of the of the number of contributors can lead to a likelihood ratio that's unfavorable to the defendant  review the article obviously in the course of this case and if the court were to do that it will show that a minor contributor didn't go it doesn't show that it went as low as it did here in terms of faulty it shows false positives due to  illegal sharing and masking which is something that dr crane testified about it shows problems during processing causing exclusions of known contributors to that point i wanted to highlight for the court that the government  review on the one hand which suggests it's very case specific district court judge to sift records specific as to that case and then on the other hand having a system in which we're processing criminal cases where there is some across cases because it doesn't seem to me storm x is going away i mean even the p test report supports the twenty percent and then says that will go lower as you get more you know expertise in implementing it this is what troubles me about your pushback on the last three factors i would have thought it's time for a court of appeals to say this kind of thing is fine but you know drill down in the validation drill down on exactly how it's used in this case because that the that should be case that works fine for me but you want to be suggesting that storm hits isn't ready for prime time and a very large swath of cases that don't seem like an abuse of discretion is it will turn on the discretion the unelected conscience of each district court judge and circuit that is not healthy when it comes to equivalent complication of the law across a lot of companies and a lot of individuals what mainstream samples quality samples that is not something that the court needs to address and it work getting there when it costs a little over twenty percent over that's what you're just it's right that i think is probably the best bright line guide if that because has not changed its position that was not simply a decision of one individual this involves the community the community that is involved that works on this on a regular basis and even in response to the thirty one lab study and dr john bolton's sit down with that community they didn't change their position they just need a little bit more this is acceptable for a simpler  mixtures i'd like to elicit i think that us feel less than the district of minnesota's decision and two thousand and twenty in this case illustrates how this is not difficult for a court to do the court distinguish the type of sample and that particular case from the low template d n a sample here and gets to enter knowing that there are six times more d n a in that particular case and a greater contribution at large to buy from the person of interest this is not a difficult assessment to do it on a case by case basis exactly my orientation of the case i want to say the last three factors plainly support storm x and i want to focus on the st factor invalidation and validation is the key reason not the only reason but the key reason if they were to have given the law the likelihood ratios and demonstrated that they repeated those tests such that there was some percentage and two to the likelihood ratio that was generated we're still only talking about five samples on this old version and this is all done without any sort of standard it's all done without any sort of agreement in the community about how to interpret that mixture and how to validate a prob nothing is binding so even at this this particular factor the testing doesn't get off the ground without without those standards being in play it sounds like you're saying the council if there is no agreement in the community respectfully there is an agreement that for the simpler mixers star makes us appropriate we reach that there may be a few outliers that don't agree that we want that think start access to risky for any sort of mixture but that's very different from what we have here about your concern about this being on the outer edges with the communities accent being accounted for through the lens of whether they've met their burden of proof to show beyond a reasonable doubt this is the right technical testimony over nearly nearly two years opposing camps of experts both domestic and international explaining concepts scientific issues and conceding that subjectivity added variability all about controlling sample this is not a simple thing and there is a real unexpected you know the software works it's been tested it's been published multiple times you know the software works or is or something it's a gut check raise there's something different about our laboratory so they're not seeking to do all these extreme experiments to figure out you know where the error is and and so you know when you look at what they did here they tested in your original summary of the internal validation study they tested down to twenty six people grams underneath the forty nine p grams here they tested down to four percent for the percent contributed by minor contributor well you know covering the seven percent here and so that was the gut check to say ok you know these samples are clearly within the realm of what starbucks can handle base revile well no it did your honor because the government introduced a letter showing exhibit exhibit forty showing that those experiments have been done but then you know dr crane says well actually no it's he said it's on point it's what i was looking for but now i actually want to see the electrified programs the likelihood ratio and if we go back and do it again there's there's just going to be more of that because you can always pull on the strength little bit further why why are you so sure about that why not give them the benefit of the doubt why what's the problem about because you are saying the state did not have a chance to produce those numbers right correct will tell me why it's such a full sorry and to let the state produce those don't be one thing but i think if you look at the history of the litigation here you know the arguments of changed about half a dozen times in terms of what we're talking about you know don't blame them for that i mean this is unusual stuff i'm not i'm not going to be a skeptic about that but if you have a decision from the court of appeals that says the last three or factors favor store minutes they were under appreciated by the just a court judge it was still fair for the district court judge to be focused on internal delegation it does seem fair and appropriate particularly the outer edges we just think the government should have been given a chance to produce the numbers in a while i am not sure why you're so reluctant to think that's the way to handle this well  i think it's because of just what i said before the debate development of the proceeding i mean obviously that that would be a don't trust the district court judge is what's going on this case back to the whole abus discretion problem each district court judge looking at this differently and you have you know twenty five different store makes rules well your honor i i think that's mostly right i wouldn't say that i don't trust the judge i do trust the judge i think that the judge here was was caught up with him was led astray by the power of the evidence i think some of the remarks at the hearing and also the fact in the opinion of the court a couple times mentions you know in criminal cases we need a higher standard of life and liberty isn't going to me get this let me put the shoe on the other foot oh so the evidence shows exonerated the defense here that none of it it's you know conclusion of that the defendant's d n a is not present or not or he wasn't a contributor hearns argument b then in terms of what the standard is for validation well when right when the validation when the validation supports the work that was done in the case of the government thinks all the evidence shows it was done here jeff bush just wants to be more specific actually put the shoe on the other foot in this case it would suffice to be absolutely absolutely yes but i but is there any case where it would not  well sure i mean i think you know if we want to take a hypothetical based on this case the government's position has been and you know the validation study of the summary clearly support it down to four percent or twenty six people grams ignoring this interaction fact which you know the government think that argument has been contrived but i think in this case if the michigan state police were saying well we tested a sample that was two percent of the mixture with only five pizza grams we wouldn't be offering that  e now if the science develops that may change right i mean they may rebound at lower levels but i'm but based on this record yes there were boundaries beyond which we would support the work but you know i mean the reason we're unlikely to see the case is because the laboratory is not going to bring that to us in the st place because they want to stay within the parameters they validate and they can ask what counts on the other side suggested that five cases are really not that much to you do you have a view of when you're getting down to or at actually at any level how much validation should be done how many tests to confirm accuracy should be done in order to make sure that it's reliably working is there and is there somebody in the more general the mat in terms of what you have to do in you don't have testimony on them for what those thresholds are but i think it's a bit of a red herring to say well there were only five experiments those levels because of course dr crane of this deposition said i just want to see that they've tested where the samples are are are that well and they said ok we'll write some of them up and i think there is even a footnote that says well there are even more of them out there but he was  and there's some discussion about the degree of reproducibility but it is a summary of the validation study where they're talking about what they did in prose and setting for some of the data they're not they're not sort of outlining all the data that the laboratory has the government thinks that's appropriate because that's not what requires job or doesn't require you know the laboratory to issue you a one thousand page validation study with all the data you know betts in their in their system all right