diff --git a/Abnormal-CT-Generation-MultiDisease/LICENSE b/Abnormal-CT-Generation-MultiDisease/LICENSE new file mode 100644 index 0000000000000000000000000000000000000000..f288702d2fa16d3cdf0035b15a9fcbc552cd88e7 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LICENSE @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/__init__.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..83f733a0e56b0e547b7d6e004a2b3bf6eedeced6 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__init__.py @@ -0,0 +1,4 @@ + +from .models.autoencoder import LeanVAE +from .models.autoencoder_pl import AutoEncoderEngine + diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/__init__.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..ec2f973c504b98bcc12fb2c0ea58259bb0e7a0be Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/__init__.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/__init__.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/__init__.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..2a88cfb537ae399ec3c8e29b934700b2b70710d3 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/__init__.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/__init__.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/__init__.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..a37f4030474c8e95d48afd2700e293bd92f0bcd2 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/__init__.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/data.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/data.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..1df8108349509b985289d07ee0df9423be4d90d2 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/data.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/data.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/data.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..062096721fbbd51dfd70d2f8b4bd5c14a460ad1f Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/__pycache__/data.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/ckpts/LeanVAE-dim16.ckpt b/Abnormal-CT-Generation-MultiDisease/LeanVAE/ckpts/LeanVAE-dim16.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..1edb223dac2ce1fa35e186ed038f7be19bff827c --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/ckpts/LeanVAE-dim16.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c1d65765e44ced040a43a0bb7084a936e5b3862d21df4a9fd13580508cd1ecb +size 159199850 diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/data.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/data.py new file mode 100644 index 0000000000000000000000000000000000000000..03cd3720ab98e9d147bf278c94dd3e19cadc6bc2 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/data.py @@ -0,0 +1,466 @@ +import os +import os.path as osp +import math +import random +import argparse +import numpy as np +from PIL import Image +from torch.utils.data import BatchSampler, Dataset, Sampler +import torch +import torch.utils.data as data +import torch.nn.functional as F +import torch.distributed as dist +from torchvision.datasets.video_utils import VideoClips +import pytorch_lightning as pl +from typing import TypeVar, Optional, Iterator, List +from collections import Counter, defaultdict +from decord import VideoReader +from .utils.video_utils import VideoNorm + +try: + from torchvision.transforms import InterpolationMode + + def _pil_interp(method): + if method == 'bicubic': + return InterpolationMode.BICUBIC + elif method == 'lanczos': + return InterpolationMode.LANCZOS + elif method == 'hamming': + return InterpolationMode.HAMMING + else: + # default bilinear, do we want to allow nearest? + return InterpolationMode.BILINEAR + + + import timm.data.transforms as timm_transforms + + timm_transforms._pil_interp = _pil_interp +except: + from timm.data.transforms import _pil_interp + +class MultiSizeVideoDataset(data.Dataset): + """ A flexible dataset for loading videos of different resolutions stored in a structured format. + This dataset reads video file paths from text files, where each file corresponds to a specific resolution (e.g., `256x256`). + Returns BCTHW videos in the range [-0.5, 0.5] """ + def __init__(self, data_list, data_folder=None, sequence_length=17, train=True, sample_rate=1, dynamic_sample=False): + """ + Args: + data_list (str): Path to the folder containing text files with video paths. + data_folder (Optional[str]): Root folder where videos are stored (if paths in data_list are relative). + + sequence_length: length of extracted video sequences + """ + super().__init__() + self.train = train + self.data_folder = data_folder + self.sequence_length = sequence_length + self.dynamic_sample = dynamic_sample + self.sample_rate = sample_rate + + lengths = [] + annotations = [] + for dir in os.listdir(data_list): + file_path = os.path.join(data_list, dir) + with open(file_path) as f: + annotation = [ann.strip() for ann in f.readlines()] + annotations.extend(annotation) + lengths.extend([dir] * len(annotation)) + + self.annotations = annotations + self.lengths = lengths + + self.norm = VideoNorm() + + def __len__(self): + return len(self.annotations) + + + def __getitem__(self, idx): + + video_path = self.annotations[idx] if self.data_folder is None else os.path.join(self.data_folder, self.annotations[idx]) + try: + decord_vr = VideoReader(video_path) + total_frames = len(decord_vr) + except Exception as e: + raise RuntimeError(f"Failed to read video: {video_path}. Error: {e}") + + if self.dynamic_sample: + sample_rate = random.randint(1, self.sample_rate) + else: + sample_rate = self.sample_rate + + required_frames = self.sequence_length * sample_rate + if total_frames < self.sequence_length: + raise RuntimeError(f"Video {video_path} has only {total_frames} frames, but {self.sequence_length} are required.") + + if total_frames < required_frames: + sample_rate = 1 + required_frames = self.sequence_length + + start_frame_ind = random.randint(0, max(0, total_frames - required_frames)) + end_frame_ind = min(start_frame_ind + required_frames, total_frames) + frame_indice = np.linspace( + start_frame_ind, end_frame_ind - 1, self.sequence_length, dtype=int + ) + + video_data = decord_vr.get_batch(frame_indice).asnumpy() + video_data = torch.from_numpy(video_data).float() + video_data = video_data.permute(0, 3, 1, 2) + + video = self.norm(video_data).permute(1, 0, 2, 3) + return {"video": video} + +class MultiFilesBatchVideoSampler(BatchSampler): + """A sampler wrapper for grouping videos within same folder into a same batch. + Args: + sampler (Sampler): Base sampler. + dataset (Dataset): Dataset providing data information. + batch_size (int): Size of mini-batch. + drop_last (bool): If ``True``, the sampler will drop the last batch if + its size would be less than ``batch_size``. + aspect_ratios (dict): The predefined aspect ratios. + """ + def __init__(self, + sampler: Sampler, + dataset: Dataset, + batch_size: int, + train_folder: str = None, + drop_last: bool = False + ) -> None: + if not isinstance(sampler, Sampler): + raise TypeError('sampler should be an instance of ``Sampler``, ' + f'but got {sampler}') + if not isinstance(batch_size, int) or batch_size <= 0: + raise ValueError('batch_size should be a positive integer value, ' + f'but got batch_size={batch_size}') + self.sampler = sampler + self.dataset = dataset + self.train_folder = train_folder + self.batch_size = batch_size + self.drop_last = drop_last + self.bucket = {file_name: [] for file_name in os.listdir(self.train_folder)} + + #{file_name: [list(os.listdir(os.path.join(self.train_folder, file_name)))] for file_name in os.listdir(self.train_folder)} + self.idx2file = [] + + + def __iter__(self): + for idx in self.sampler: + file_name = self.idx2file[idx] + self.bucket[file_name].append(idx) + bucket = self.bucket[file_name] + bucket.append(idx) + # yield a batch of indices in the same aspect ratio group + if len(bucket) == self.batch_size: + yield bucket[:] + del bucket[:] + + +def group_data_fun(lengths, generator=None): + # counter is decrease order + counter = Counter(lengths) # counter {'1x256x256': 3, ''} lengths ['1x256x256', '1x256x256', '1x256x256', ...] + grouped_indices = defaultdict(list) + for idx, item in enumerate(lengths): # group idx to a list + grouped_indices[item].append(idx) + + grouped_indices = dict(grouped_indices) # {'1x256x256': [0, 1, 2], ...} + sorted_indices = [grouped_indices[item] for (item, _) in sorted(counter.items(), key=lambda x: x[1], reverse=True)] + + # shuffle in each group + shuffle_sorted_indices = [] + for indice in sorted_indices: + shuffle_idx = torch.randperm(len(indice), generator=generator).tolist() + shuffle_sorted_indices.extend([indice[idx] for idx in shuffle_idx]) + return shuffle_sorted_indices + +def last_group_data_fun(shuffled_megabatches, lengths): + # lengths ['1x256x256', '1x256x256', '1x256x256' ...] + re_shuffled_megabatches = [] + # print('shuffled_megabatches', len(shuffled_megabatches)) + for i_megabatch, megabatch in enumerate(shuffled_megabatches): + re_megabatch = [] + for i_batch, batch in enumerate(megabatch): + assert len(batch) != 0 + + len_each_batch = [lengths[i] for i in batch] # ['1x256x256', '1x256x256'] + idx_length_dict = dict([*zip(batch, len_each_batch)]) # {0: '1x256x256', 100: '1x256x256'} + count_dict = Counter(len_each_batch) # {'1x256x256': 2} or {'1x256x256': 1, '1x768x256': 1} + if len(count_dict) != 1: + sorted_by_value = sorted(count_dict.items(), key=lambda item: item[1]) # {'1x256x256': 1, '1x768x256': 1} + # import ipdb;ipdb.set_trace() + # print(batch, idx_length_dict, count_dict, sorted_by_value) + pick_length = sorted_by_value[-1][0] # the highest frequency + candidate_batch = [idx for idx, length in idx_length_dict.items() if length == pick_length] + random_select_batch = [random.choice(candidate_batch) for i in range(len(len_each_batch) - len(candidate_batch))] + # print(batch, idx_length_dict, count_dict, sorted_by_value, pick_length, candidate_batch, random_select_batch) + batch = candidate_batch + random_select_batch + # print(batch) + + for i in range(1, len(batch)-1): + # if not lengths[batch[0]] == lengths[batch[i]]: + # print(batch, [lengths[i] for i in batch]) + # import ipdb;ipdb.set_trace() + assert lengths[batch[0]] == lengths[batch[i]] + re_megabatch.append(batch) + re_shuffled_megabatches.append(re_megabatch) + + + # for megabatch, re_megabatch in zip(shuffled_megabatches, re_shuffled_megabatches): + # for batch, re_batch in zip(megabatch, re_megabatch): + # for i, re_i in zip(batch, re_batch): + # if i != re_i: + # print(i, re_i) + return re_shuffled_megabatches + +def split_to_even_chunks(megabatch, lengths, world_size, batch_size): + """ + Split a list of indices into `chunks` chunks of roughly equal lengths. + """ + # batch_size=2, world_size=2 + # [1, 2, 3, 4] -> [[1, 2], [3, 4]] + # [1, 2, 3] -> [[1, 2], [3]] + # [1, 2] -> [[1], [2]] + # [1] -> [[1], []] + chunks = [megabatch[i::world_size] for i in range(world_size)] + + pad_chunks = [] + for idx, chunk in enumerate(chunks): + if batch_size != len(chunk): + assert batch_size > len(chunk) + if len(chunk) != 0: # [[1, 2], [3]] -> [[1, 2], [3, 3]] + chunk = chunk + [random.choice(chunk) for _ in range(batch_size - len(chunk))] + else: + chunk = random.choice(pad_chunks) # [[1], []] -> [[1], [1]] + print(chunks[idx], '->', chunk) + pad_chunks.append(chunk) + return pad_chunks + +def get_length_grouped_indices(lengths, batch_size, world_size, gradient_accumulation_size, initial_global_step, generator=None, group_data=False, seed=42): + # We need to use torch for the random part as a distributed sampler will set the random seed for torch. + if generator is None: + generator = torch.Generator().manual_seed(seed) # every rank will generate a fixed order but random index + # print('lengths', lengths) + + if group_data: + indices = group_data_fun(lengths, generator) + else: + indices = torch.randperm(len(lengths), generator=generator).tolist() + + megabatch_size = world_size * batch_size + megabatches = [indices[i: i + megabatch_size] for i in range(0, len(lengths), megabatch_size)] + + megabatches_len = [[lengths[i] for i in megabatch] for megabatch in megabatches] + + megabatches = [split_to_even_chunks(megabatch, lengths, world_size, batch_size) for megabatch in megabatches] + + split_to_even_chunks_len = [[[lengths[i] for i in batch] for batch in megabatch] for megabatch in megabatches] + + indices_mega = torch.randperm(len(megabatches), generator=generator).tolist() + # print(f'rank {accelerator.process_index} seed {seed}, len(megabatches) {len(megabatches)}, indices_mega, {indices_mega[:50]}') + shuffled_megabatches = [megabatches[i] for i in indices_mega] + shuffled_megabatches_len = [[[lengths[i] for i in batch] for batch in megabatch] for megabatch in shuffled_megabatches] + # print(f'\nrank {accelerator.process_index} sorted shuffled_megabatches_len', shuffled_megabatches_len[0], shuffled_megabatches_len[1], shuffled_megabatches_len[-2], shuffled_megabatches_len[-1]) + + # import ipdb;ipdb.set_trace() + # print('shuffled_megabatches', len(shuffled_megabatches)) + if group_data: + shuffled_megabatches = last_group_data_fun(shuffled_megabatches, lengths) + group_shuffled_megabatches_len = [[[lengths[i] for i in batch] for batch in megabatch] for megabatch in shuffled_megabatches] + # print(f'\nrank {accelerator.process_index} group_shuffled_megabatches_len', group_shuffled_megabatches_len[0], group_shuffled_megabatches_len[1], group_shuffled_megabatches_len[-2], group_shuffled_megabatches_len[-1]) + + + #initial_global_step = initial_global_step * gradient_accumulation_size #todo + + shuffled_megabatches = shuffled_megabatches[initial_global_step:] + #print(f'Skip the data of {initial_global_step} step!') + + return [batch for megabatch in shuffled_megabatches for batch in megabatch] + +class LengthGroupedSampler(Sampler): + r""" + Sampler that samples indices in a way that groups together features of the dataset of roughly the same length while + keeping a bit of randomness. + """ + + def __init__( + self, + batch_size: int, + world_size: int, + gradient_accumulation_size: int = 1, + initial_global_step: int = 0, + lengths: Optional[List[int]] = None, + group_data=False, + generator=None, + rank: Optional[int] = None, + seed: int = 0, + ): + if lengths is None: + raise ValueError("Lengths must be provided.") + + self.batch_size = batch_size + self.world_size = world_size + self.initial_global_step = initial_global_step + self.gradient_accumulation_size = gradient_accumulation_size + self.lengths = lengths + self.group_data = group_data + self.generator = generator + + self.rank = rank + self.epoch = 0 + + self.seed = seed + + megabatch_size = self.batch_size * self.world_size + self.num_samples = ((len(lengths) + megabatch_size - 1) // megabatch_size ) * self.batch_size #todo + #self.num_samples = self.num_samples #- self.initial_global_step * self.batch_size * self.gradient_accumulation_size + # print('self.lengths, self.initial_global_step, self.batch_size, self.world_size, self.gradient_accumulation_size', + # len(self.lengths), self.initial_global_step, self.batch_size, self.world_size, self.gradient_accumulation_size) + + def __len__(self): + return self.num_samples + + def __iter__(self): + g = torch.Generator() + g.manual_seed(self.seed + self.epoch) + megabatch_indices = get_length_grouped_indices(self.lengths, self.batch_size, self.world_size, + self.gradient_accumulation_size, self.initial_global_step, + group_data=self.group_data, generator=g) + + # subsample + indices = [i for batch in megabatch_indices[self.rank::self.world_size] for i in batch] + assert len(indices) == self.num_samples + + return iter(indices) + + def set_epoch(self, epoch: int) -> None: + r""" + Set the epoch for this sampler. + + When :attr:`shuffle=True`, this ensures all replicas + use a different random ordering for each epoch. Otherwise, the next iteration of this + sampler will yield the same ordering. + + Args: + epoch (int): Epoch number. + """ + self.epoch = epoch + + +class VideoData(pl.LightningDataModule): + def __init__(self, args): + super().__init__() + self.args = args + + def _dataset(self, train): + datasets = [] + for dataset_path, train_list, val_list in zip(self.args.data_path, self.args.train_datalist, self.args.val_datalist): + + dataset = MultiSizeVideoDataset(data_folder=dataset_path, data_list=train_list if train else val_list, sequence_length=self.args.sequence_length, + train=train, sample_rate=self.args.sample_rate, dynamic_sample=self.args.dynamic_sample) + datasets.append(dataset) + return datasets + + def _dataloader(self, train, steps = 0, batch_size = None): + dataset = self._dataset(train) + if isinstance(self.args.batch_size, int): + self.args.batch_size = [self.args.batch_size] + self.batch_size = self.args.batch_size if batch_size is None else batch_size + assert len(dataset) == len(self.args.batch_size) + dataloaders = [] + for dset, d_batch_size in zip(dataset, self.batch_size): + if dist.is_initialized(): + sampler = LengthGroupedSampler( + batch_size=d_batch_size, + world_size=dist.get_world_size(), + gradient_accumulation_size=1, + initial_global_step=steps if train else 0, + lengths=dset.lengths, + group_data=True, + rank = dist.get_rank() + ) + else: + sampler = None + + dataloader = data.DataLoader( + dset, + batch_size=d_batch_size, + num_workers=self.args.num_workers if train else 0, + pin_memory=False, + sampler=sampler, + ) + + dataloaders.append(dataloader) + + return dataloaders + + def train_dataloader(self): + return self._dataloader(True) + + def val_dataloader(self): + return self._dataloader(False)[0] + + + @staticmethod + def add_data_specific_args(parent_parser): + parser = argparse.ArgumentParser(parents=[parent_parser], add_help=False) + parser.add_argument('--data_path', type=str, nargs="+", default=['']) + parser.add_argument('--train_datalist', type=str, nargs="+", default=['./video/kinetics-dataset/train/datapath']) + parser.add_argument('--val_datalist', type=str, nargs="+", default=['./video/kinetics-dataset/val/datapath']) + + parser.add_argument('--sequence_length', type=int, default=17) + parser.add_argument('--sample_rate', type=int, default=1, + help='Frame sampling rate') + parser.add_argument('--dynamic_sample', action='store_true', + help='Enable dynamic sampling rate') + + parser.add_argument('--batch_size', type=int, nargs="+", default=[5]) + parser.add_argument('--num_workers', type=int, default=8) + return parser + +if __name__ == "__main__": + import os + def lines(file_path): + with open(file_path, 'r') as file: + return sum(1 for line in file) + train_folder ='./kinetics-dataset/datapath' + lengths_dict = {file_name: lines(os.path.join(train_folder, file_name)) for file_name in os.listdir(train_folder)} + lengths = [] + for k, v in lengths_dict.items(): + lengths += [k] * min(v, 50) #(v % 7) + world_size = 4 + sampler = [] + batch_size = 10 + for rank in range(world_size): + sampler.append(LengthGroupedSampler( + batch_size=batch_size, + world_size=world_size, + gradient_accumulation_size=1, + initial_global_step=0, + lengths=lengths, + group_data=True, + rank = rank + )) + + + with open('./sampler.txt', 'w') as f: + for epoch in range(5): + rank_idx = {} + bk = [] + print(f'epoch -------------------------------------- {epoch} ----------------------------------------------------', file=f) + for rank in range(world_size): + sl = sampler[rank] + sl.set_epoch(epoch) + for i in iter(sl): + bk.append(i) + if len(bk) == batch_size: + rank_idx.setdefault(f'rank_{rank}', []) + rank_idx[f'rank_{rank}'].append(bk) + bk = [] + for num in range(5): + print('*'*5 + f'steps {num}' + '*'*5, file=f) + for rank, bk in rank_idx.items(): + print(f'rank {rank}: {bk[num]}', file=f) + print([lengths[i] for i in bk[num]], file=f) + + + exit() diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..10cb37bd3a9ba66c51eb294606b9bb4cf6a1ad03 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..a1c1426a7ca7748feba3839a12d6579e8c37b9bb Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..f2197a347ce11a2a4f2161bcbecf81aac9d18d6d Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder_pl.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder_pl.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..b495b4548c4393c00b46efcbac85b64bae083a4f Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder_pl.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder_pl.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder_pl.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..f10c8db09d6dac885bdff1e92660a9fc61fcdeee Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder_pl.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder_pl.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder_pl.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..246261368e5c5aa58cbf5b0c42397d5d35fa2b18 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/__pycache__/autoencoder_pl.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/autoencoder.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/autoencoder.py new file mode 100644 index 0000000000000000000000000000000000000000..55e028a445e9a4fcf4774d555c4bcecfea3b731c --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/autoencoder.py @@ -0,0 +1,186 @@ +import argparse +import torch +import torch.distributed +import torch.nn as nn +import torch.nn.functional as F +from einops import rearrange +from ..modules import DiagonalGaussianDistribution, Encoder_Arch, Decoder_Arch, ISTA +from ..utils.patcher_utils import Patcher, UnPatcher + +class LeanVAE(nn.Module): + def __init__(self, args): + super().__init__() + self.args = args + self.embedding_dim = args.embedding_dim + + self.latent_bottleneck = ISTA(points_num=args.embedding_dim, out_num=args.latent_dim, iter_num=args.ista_iter_num, layer_num=args.ista_layer_num) + + self.dwt = Patcher() + self.idwt = UnPatcher() + + self.encoder = Encoder_Arch(l_dim = args.l_dim, h_dim = args.h_dim, sep_num_layer = args.sep_num_layer, fusion_num_layer = args.fusion_num_layer) + self.decoder = Decoder_Arch(l_dim = args.l_dim, h_dim = args.h_dim, sep_num_layer = args.sep_num_layer, fusion_num_layer = args.fusion_num_layer) + + self.std_layer = nn.Linear(args.embedding_dim, args.latent_dim) + + self.tile_inference = False + self.chunksize_enc = args.chunksize_enc if hasattr(args, 'chunksize_enc') and args.chunksize_enc else 5 + self.chunksize_dec = args.chunksize_dec if hasattr(args, 'chunksize_dec') and args.chunksize_dec else 5 + if args.use_tile_inference: + self.set_tile_inference(True) + else: + self.set_tile_inference(False) + + def _set_first_chunk(self, is_first_chunk=True): + for module in self.modules(): + if hasattr(module, 'is_first_chunk'): + module.is_first_chunk = is_first_chunk + + def set_tile_inference(self, tile_inference=False): + for module in self.modules(): + if hasattr(module, 'tile_inference'): + module.tile_inference = tile_inference + + def _build_chunk_index(self, T = 17, mtype = 'enc'): + start_end = [] + if mtype == 'enc': + chunksize = self.chunksize_enc + else: + chunksize = self.chunksize_dec + if T >= chunksize : + start_end.append((0, chunksize)) + start_idx = chunksize + else: + assert T < chunksize + + for i in range(start_idx, T, chunksize-1): + end_idx = min(i + chunksize -1, T) + start_end.append((i, end_idx)) + return start_end + + def encode(self, x): + ndim = x.ndim + if ndim == 4: + x = x.unsqueeze(2) + self.set_tile_inference(False) + + if self.tile_inference: + z = [] + chunk_indexs = self._build_chunk_index(T=x.shape[2], mtype='enc') + for idx, (start, end) in enumerate(chunk_indexs): + if idx == 0: + self._set_first_chunk(True) + else: + self._set_first_chunk(False) + + x_dwt = self.dwt(x[:, :, start:end]) + p = self.encoder.encode(x=x_dwt) + z.append(self.latent_bottleneck.sample(p)) + z = torch.cat(z, dim = 1) + else: + x_dwt = self.dwt(x) + p = self.encoder.encode(x=x_dwt) + z = self.latent_bottleneck.sample(p) + + z = rearrange(z, 'b t h w d -> b d t h w') + return z + + def decode(self, z, is_image = False): + z = rearrange(z, 'b d t h w -> b t h w d') + if is_image: + self.set_tile_inference(False) + if self.tile_inference: + x_recon = [] + chunk_indexs = self._build_chunk_index(T=z.shape[1], mtype='dec') + for idx, (start, end) in enumerate(chunk_indexs): + if idx == 0: + self._set_first_chunk(True) + else: + self._set_first_chunk(False) + p_rec = self.latent_bottleneck.recon(z[:, start:end]) + x_dwt_rec = self.decoder.decode(p_rec, is_image=is_image) + + x_recon.append(self.idwt(x=x_dwt_rec)) + x_recon = torch.cat(x_recon, dim = 2) + else: + p_rec = self.latent_bottleneck.recon(z) + x_dwt_rec = self.decoder.decode(p_rec, is_image=is_image) + + x_recon = self.idwt(x=x_dwt_rec) + + return x_recon + + + + @torch.no_grad() + def inference(self, x): + if x.ndim == 4 : + is_image = True + else: + is_image = False + assert x.shape[2] % 4 == 1, f"Expected frame_num % 4 == 1, but got {x.shape[2] % 4}" + + z = self.encode(x) + x_recon = self.decode(z, is_image=is_image) + + if is_image: + x = x.squeeze(2) + return x, x_recon + + def forward(self, x, log_image=False): + x_dwt = self.dwt(x) + p = self.encoder(x=x_dwt) + z_mean = self.latent_bottleneck.sample(p) + z_std = self.std_layer(p) + + posterior = DiagonalGaussianDistribution(parameters=(z_mean, z_std)) + z = posterior.sample() + p_rec = self.latent_bottleneck.recon(z) + + x_dwt_rec = self.decoder(p_rec) #b c t h w + + + x_recon = self.idwt(x=x_dwt_rec) + + if log_image: + return x, x_recon + + return x, x_recon, x_dwt, x_dwt_rec, posterior + + + @classmethod + def load_from_checkpoint(cls, ckpt_path, device="cpu", strict=False): + """ Load model from checkpoint, initializing args and state_dict """ + checkpoint = torch.load(ckpt_path, map_location=device) + + if "args" not in checkpoint: + raise ValueError("Checkpoint does not contain 'args'. Ensure the checkpoint is saved correctly.") + + args = argparse.Namespace(**checkpoint["args"]) + + model = cls(args) + if "state_dict" in checkpoint: + msg = model.load_state_dict(checkpoint["state_dict"], strict=strict) + print(f"Successfully loaded weights from {ckpt_path}, {msg}") + return model + + @staticmethod + def add_model_specific_args(parent_parser): + parser = argparse.ArgumentParser(parents=[parent_parser], add_help=False) + + # Model architecture parameters + parser.add_argument("--embedding_dim", type=int, default=512, help="Dimension of the embedding space.") + parser.add_argument("--latent_dim", type=int, default=4, help="Dimension of the latent channel.") + parser.add_argument("--ista_iter_num", type=int, default=2, help="Number of iterations in ISTA latent bottleneck.") + parser.add_argument("--ista_layer_num", type=int, default=2, help="Number of layers in ISTA latent bottleneck.") + + parser.add_argument("--l_dim", type=int, default=128) + parser.add_argument("--h_dim", type=int, default=384) + parser.add_argument("--sep_num_layer", type=int, default=2, help="Number of separate processing layers in encoder/decoder.") + parser.add_argument("--fusion_num_layer", type=int, default=4, help="Number of fusion layers in encoder/decoder.") + + # Tiling inference (for memory-efficient processing) + parser.add_argument("--use_tile_inference", action="store_true", help="Enable tiling inference to process video in chunks.") + parser.add_argument("--chunksize_enc", type=int, default=9, help="Number of frames per chunk during tiled encoding.") + parser.add_argument("--chunksize_dec", type=int, default=5, help="Number of frames per chunk during tiled decoding.") + return parser diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/autoencoder_pl.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/autoencoder_pl.py new file mode 100644 index 0000000000000000000000000000000000000000..96b58ae98017752e18b096e985e18650b65668be --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/models/autoencoder_pl.py @@ -0,0 +1,232 @@ +import argparse +import numpy as np +from PIL import Image +import pytorch_lightning as pl +import torch +import torch.distributed +import torch.nn as nn +import torch.nn.functional as F +from timm.scheduler.cosine_lr import CosineLRScheduler +from timm.models.layers import trunc_normal_ +from .autoencoder import LeanVAE +from ..modules import LPIPS +from ..utils.gan_loss import AdversarialLoss + +class AutoEncoderEngine(pl.LightningModule): + def __init__(self, args, data): + super().__init__() + self.args = args + self.video_data = data + + self.autoencoder = LeanVAE(args=args) + + self.automatic_optimization = False + self.kl_weight = args.kl_weight + self.discriminator_iter_start = args.discriminator_iter_start + + self.perceptual_weight = args.perceptual_weight + self.l1_weight = args.l1_weight + + self.automatic_optimization = False + self.grad_clip_val = args.grad_clip_val + + if not hasattr(args, "grad_clip_val_disc"): + args.grad_clip_val_disc = 1.0 + + self.grad_clip_val_disc = args.grad_clip_val_disc + + self.apply(self._init_weights) + self.perceptual_model = LPIPS().eval() + self.perceptual_model.requires_grad_(False) + self.gan_loss = AdversarialLoss(disc_weight=args.disc_weight) + self.save_hyperparameters() + + def _init_weights(self, m): + if isinstance(m, nn.Linear): + trunc_normal_(m.weight, std=.02) + if isinstance(m, nn.Linear) and m.bias is not None: + nn.init.constant_(m.bias, 0) + elif isinstance(m, nn.LayerNorm): + if m.bias is not None: + nn.init.constant_(m.bias, 0) + if m.weight is not None: + nn.init.constant_(m.weight, 1.0) + + elif isinstance(m, nn.Conv3d) or isinstance(m, nn.Conv2d): + nn.init.xavier_uniform_(m.weight) + nn.init.zeros_(m.bias) + + + def forward(self, x, optimizer_idx=None, x_recon = None, log_image=False): + if log_image: + return self.autoencoder(x, log_image) + + if optimizer_idx == 1: + discloss = self.gan_loss(inputs=x, reconstructions=x_recon, optimizer_idx=1) + self.log("train/discloss", discloss, prog_bar=True, logger=True, on_step=True, on_epoch=True) + return discloss + + elif optimizer_idx == 0: + assert x.ndim == 5 + B, C, T, H, W = x.shape + x, x_recon, x_dwt, x_dwt_rec, posterior = self.autoencoder(x) + recon_loss = F.l1_loss(x_recon, x)* self.l1_weight + kl_loss = posterior.kl() + kl_loss = torch.sum(kl_loss) / kl_loss.shape[0] * self.kl_weight + + g_loss = 0.0 + if self.global_step >= self.discriminator_iter_start: + g_loss = self.gan_loss(x, x_recon, optimizer_idx=0) + self.log("train/g_loss", g_loss, prog_bar=True, logger=True, on_step=True, on_epoch=True) + + recon_loss_low = (F.l1_loss(x_dwt_rec[0][:, :3], x_dwt[0][:, :3]) + F.l1_loss(x_dwt_rec[1][:, :3], x_dwt[1][:, :3])) * self.l1_weight * 0.05 + recon_loss_high = (F.l1_loss(x_dwt_rec[0][:, 3:], x_dwt[0][:, 3:])+ F.l1_loss(x_dwt_rec[1][:, 3:], x_dwt[1][:, 3:])) * self.l1_weight * 0.1 + + k = 4 + valid_start_indices = torch.tensor([x for x in range(T - k + 1) if x % 4 == 1]) + start_idx = valid_start_indices[torch.randint(0, len(valid_start_indices), (B,))] + frame_idx = start_idx.unsqueeze(1) + torch.arange(k) + frame_idx = torch.cat((torch.zeros((B, 1), dtype=torch.int), frame_idx), dim=1).to(self.device) + + frame_idx_selected = frame_idx.reshape(-1, 1, k+1, 1, 1).repeat(1, C, 1, H, W) + frames = torch.gather(x, 2, frame_idx_selected) + frames_recon = torch.gather(x_recon, 2, frame_idx_selected) + frames = frames.permute(0, 2, 1, 3, 4).contiguous().view(-1, 3, H, W) + frames_recon = frames_recon.permute(0, 2, 1, 3, 4).contiguous().view(-1, 3, H, W) + perceptual_loss = self.perceptual_model(frames, frames_recon).mean() * self.perceptual_weight + + self.log("train/recon_loss", recon_loss, prog_bar=True, logger=True, on_step=True, on_epoch=True) + self.log("train/kl_loss", kl_loss, prog_bar=True, logger=True, on_step=True, on_epoch=True) + self.log("train/recon_loss_low", recon_loss_low, prog_bar=True, logger=True, on_step=True, on_epoch=True) + self.log("train/recon_loss_high", recon_loss_high, prog_bar=True, logger=True, on_step=True, on_epoch=True) + self.log("train/perceptual_loss", perceptual_loss, prog_bar=True, logger=True, on_step=True, on_epoch=True) + return perceptual_loss + recon_loss + recon_loss_low + recon_loss_high + kl_loss + g_loss, x_recon + + return perceptual_loss, recon_loss, kl_loss + + + def training_step(self, batch, batch_idx): + + x = batch[0]['video'] + cur_global_step = self.global_step + + sch1, sch2 = self.lr_schedulers() + opt1, opt2 = self.optimizers() + + cur_global_step = self.global_step + + self.toggle_optimizer(opt1, optimizer_idx=0) + loss_generator, x_recon = self.forward(x, optimizer_idx=0) + opt1.zero_grad() + self.manual_backward(loss_generator) + if self.grad_clip_val is not None: + self.clip_gradients(opt1, gradient_clip_val=self.grad_clip_val) + opt1.step() + sch1.step(cur_global_step) + self.untoggle_optimizer(optimizer_idx=0) + + if cur_global_step > self.discriminator_iter_start: + self.toggle_optimizer(opt2, optimizer_idx=1) + loss_discriminator = self.forward(x, optimizer_idx=1, x_recon=x_recon) + + opt2.zero_grad() + self.manual_backward(loss_discriminator) + + if self.grad_clip_val_disc is not None: + self.clip_gradients(opt2, gradient_clip_val=self.grad_clip_val_disc) + opt2.step() + sch2.step(cur_global_step) + self.untoggle_optimizer(optimizer_idx=1) + + + def validation_step(self, batch, batch_idx): + x = batch['video'] + perceptual_loss, recon_loss, kl_loss = self.forward(x) + self.log('val/recon_loss', recon_loss, prog_bar=True) + self.log('val/perceptual_loss', perceptual_loss, prog_bar=True) + self.log("val/kl_loss", kl_loss, prog_bar=True) + + def train_dataloader(self): + dataloaders = self.video_data._dataloader(train=True) + return dataloaders + + def val_dataloader(self): + return self.video_data._dataloader(False)[0] + + def configure_optimizers(self): + opt_ae = torch.optim.Adam(self.autoencoder.parameters(), + lr=self.args.lr, betas=(0.5, 0.9)) + + opt_disc = torch.optim.Adam( + self.gan_loss.get_trainable_parameters(), + lr=self.args.lr_min, betas=(0.5, 0.9)) + + lr_min = self.args.lr_min + train_iters = self.args.max_steps - self.discriminator_iter_start + warmup_steps = self.args.warmup_steps + warmup_lr_init = self.args.warmup_lr_init + + + sch_ae = CosineLRScheduler( + opt_ae, + lr_min = lr_min, + t_initial = train_iters, + warmup_lr_init=warmup_lr_init, + warmup_t=warmup_steps, + cycle_mul = 1., + cycle_limit=1, + t_in_epochs=True, + ) + + sch_disc = CosineLRScheduler( + opt_disc, + lr_min = lr_min , + t_initial = train_iters, + warmup_lr_init=warmup_lr_init, + warmup_t= self.args.dis_warmup_steps, + cycle_mul = 1., + cycle_limit=1, + t_in_epochs=True, + ) + + + return [opt_ae, opt_disc], [{"scheduler": sch_ae, "interval": "step"}, {"scheduler": sch_disc, "interval": "step"}] + + + + def log_videos(self, batch, **kwargs): + log = dict() + if isinstance(batch, list): + batch = batch[0] + x = batch['video'] + x, x_rec = self(x, log_image=True) + log["inputs"] = x + log["reconstructions"] = x_rec + return log + + @staticmethod + def add_model_specific_args(parent_parser): + parser = argparse.ArgumentParser(parents=[parent_parser], add_help=False) + + # training configurations + parser.add_argument('--lr', type=float, default=5e-5) + parser.add_argument('--lr_min', type=float, default=1e-5) + parser.add_argument('--warmup_steps', type=int, default=5000) + parser.add_argument('--warmup_lr_init', type=float, default=0.) + parser.add_argument('--grad_clip_val', type=float, default=1.0) + parser.add_argument('--grad_clip_val_disc', type=float, default=1.0) + + + parser.add_argument('--kl_weight', type=float, default=1e-7) + parser.add_argument('--perceptual_weight', type=float, default=4.) + parser.add_argument('--l1_weight', type=float, default=4.) + parser.add_argument('--disc_weight', type=float, default=0.2) + + # configuration for discriminator + parser.add_argument('--dis_warmup_steps', type=int, default=0) + parser.add_argument('--discriminator_iter_start', type=int, default=0) + parser.add_argument('--dis_lr_multiplier', type=float, default=1.) + + return parser + + diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__init__.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..72c215a154db4c7fd24fff62fcc897712e5339ed --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__init__.py @@ -0,0 +1,3 @@ +from .lpips import LPIPS +from .backbones import * +from .vae import DiagonalGaussianDistribution \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/__init__.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..c0a9db59b1ecc362535ce36da8d416f7806f9c89 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/__init__.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/__init__.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/__init__.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..09cbe4e68abee68cdc40b8b14b5e417883eedb2f Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/__init__.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/__init__.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/__init__.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..3ae31eb7065e912320cb7a3041761fd78e5d38b0 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/__init__.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/backbones.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/backbones.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..3200ca233ed413ee087e6af814a66b3b811a70d0 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/backbones.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/backbones.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/backbones.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..6793052f62193e2333e18cec601cbb19d6427fe8 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/backbones.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/backbones.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/backbones.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..cd68d9d0019a809c94b75dfe22d38cd52276cbb5 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/backbones.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/discriminator.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/discriminator.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..d8cbe1342209fb64698aa44553a89d5ac4c0389f Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/discriminator.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/discriminator.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/discriminator.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..52103c5adb5207910942e357085870c541b569bb Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/discriminator.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/discriminator.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/discriminator.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..8efca3d32b1f1a4661662a22381cbeacfe3fe452 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/discriminator.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/lpips.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/lpips.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..e8e9d0b8bbdc9283886746b0c5ed4a962b74003a Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/lpips.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/lpips.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/lpips.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..773a91f872c1a61d32a3aace29b4180c88dddf40 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/lpips.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/lpips.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/lpips.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..9df2ee9cf69b81debd8adf705a8c9789eb3124f9 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/lpips.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/vae.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/vae.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..1801c91e9ac28b1a044c817b4afec0a41a9438d5 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/vae.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/vae.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/vae.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..21d3a7e06e9ebed88986247b175b2b8f2d53096e Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/vae.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/vae.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/vae.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..b320fe8b5e317538c6a9ff9cab72a501dfe43e93 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/__pycache__/vae.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/backbones.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/backbones.py new file mode 100644 index 0000000000000000000000000000000000000000..606edd41e164046c71da3ba2a9d540fbe9341abc --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/backbones.py @@ -0,0 +1,402 @@ +import torch +import torch.nn as nn +import torch.nn.functional as F +from beartype import beartype +from typing import Tuple +from einops import rearrange, repeat +from einops.layers.torch import Rearrange +import numpy as np + +def exists(val): + return val is not None + +def default(val, d): + if exists(val): + return val + return d() if callable(d) else d + +class PEG3D(nn.Module): + def __init__( + self, + dim + ): + super().__init__() + self.ds_conv = nn.Conv3d(in_channels=dim, out_channels=dim, kernel_size=(3,3,3), groups = dim) + self.is_first_chunk = True + self.causal_cached = None + self.tile_inference = False + + def forward(self, x): + x = rearrange(x, 'b t h w d -> b d t h w') + if self.tile_inference: + if self.is_first_chunk: + x = F.pad(x, (1, 1, 1, 1, 2, 0), value=0.) + else: + x = F.pad(x, (1, 1, 1, 1, 0, 0), value=0.) + x = torch.concatenate((self.causal_cached, x), dim=2) + + self.causal_cached = x[:, :, -2:].clone() + else: + x = F.pad(x, (1, 1, 1, 1, 2, 0), value=0.) + x = self.ds_conv(x.contiguous()) + x = rearrange(x, 'b d t h w -> b t h w d') + return x + + +class GEGLU(nn.Module): + def forward(self, x): + x, gate = x.chunk(2, dim=-1) + return F.gelu(gate) * x + + +def ffd(dim, mult=4, dropout=0.): + inner_dim = int(mult * (2 / 3) * dim) + return nn.Sequential( + nn.LayerNorm(dim), + nn.Linear(dim, inner_dim * 2, bias=False), + GEGLU(), + nn.Dropout(dropout), + nn.Linear(inner_dim, dim, bias=False) + ) + + +class NAF(nn.Module): + def __init__(self, + num_layer, + dim, + ): + super(NAF, self).__init__() + self.num_layer = num_layer + self.dconv_layer = nn.Sequential() + self.ffd_layer = nn.Sequential() + for _ in range(num_layer): + self.ffd_layer.append(ffd(dim, 4)) + self.dconv_layer.append(PEG3D(dim)) + + def forward(self, x): + for i in range(self.num_layer): + x = self.dconv_layer[i](x) + x = self.ffd_layer[i](x) + return x + + +class ResNAF(nn.Module): + def __init__(self, + num_layer, + dim, + ): + super(ResNAF, self).__init__() + self.num_layer = num_layer + self.dconv_layer = nn.Sequential() + self.ffd_layer = nn.Sequential() + for _ in range(num_layer): + self.ffd_layer.append(ffd(dim, 4)) + self.dconv_layer.append(PEG3D(dim)) + + def forward(self, x): + for i in range(self.num_layer): + x = x + self.dconv_layer[i](x) + x = x + self.ffd_layer[i](x) + return x + + +class Encoder_Arch(nn.Module): + def __init__(self, + l_dim = 128, + h_dim = 384, + sep_num_layer = 2, + fusion_num_layer = 4, + patch_size = (2,4,4), + in_channel = 3 + ): + super(Encoder_Arch, self).__init__() + + self.is_first_chunk = True + self.tile_inference = False + + self.in_channel = in_channel + + self._build_linear_patch(in_channel=in_channel, out_channel_low=l_dim, out_channel_high=h_dim, pt=patch_size[0], ph=patch_size[1], pw=patch_size[2]) + + self.low_layer = ResNAF(num_layer=sep_num_layer, dim=l_dim) + self.high_layer = ResNAF(num_layer=sep_num_layer, dim=h_dim) + self.fusion_layer = ResNAF(num_layer=fusion_num_layer, dim=l_dim + h_dim) + + def _build_linear_patch(self, in_channel = 3, out_channel_low = 128, out_channel_high = 384, pt = 2, ph = 4, pw = 4): + patch_config = { + 'video_low': (pt, ph, pw), + 'video_high': (pt, ph, pw), + 'image_low': (1, ph, pw), + 'image_high': (1, ph, pw) + } + + for name, (t, h, w) in patch_config.items(): + if 'low' in name: + in_dim = in_channel * t * h * w + out_dim = out_channel_low + else: + out_dim = out_channel_high + in_dim = in_channel * t * h * w * 7 if 'video' in name else in_channel * t * h * w * 3 + proj = nn.Sequential( + Rearrange(f'b c (nt {t}) (nh {h}) (nw {w}) -> b nt nh nw (c {t} {h} {w})' if 'video' in name else f'b c (nh {h}) (nw {w}) -> b 1 nh nw (c {h} {w})'), + nn.Linear(in_dim, out_dim) + ) + self.add_module(f"{name}_proj", proj) + + + def _linear_patch(self, x, proj_type): + low_comp, high_comp = x[:, :self.in_channel], x[:, self.in_channel:] + return getattr(self, f"{proj_type}_low_proj")(low_comp), getattr(self, f"{proj_type}_high_proj")(high_comp) + + def forward(self, x): + xi, xv = x + xi_low, xi_high = self._linear_patch(xi, 'image') + xv_low, xv_high = self._linear_patch(xv, 'video') + + low_x = torch.cat([xi_low, xv_low], dim=1) + high_x = torch.cat([xi_high, xv_high], dim=1) + + high_x = self.high_layer(high_x) + low_x = self.low_layer(low_x) + x = torch.cat([low_x, high_x], dim=-1) + x = self.fusion_layer(x) + return x + + + + def encode(self, x): + xi, xv = x + if xi is not None and xv is not None: + xi_low, xi_high = self._linear_patch(xi, 'image') + xv_low, xv_high = self._linear_patch(xv, 'video') + + low_x = torch.cat([xi_low, xv_low], dim=1) + high_x = torch.cat([xi_high, xv_high], dim=1) + elif xi is not None: + low_x, high_x = self._linear_patch(xi, 'image') + elif xv is not None: + low_x, high_x = self._linear_patch(xv, 'video') + + high_x = self.high_layer(high_x) + low_x = self.low_layer(low_x) + x = torch.cat([low_x, high_x], dim=-1) + x = self.fusion_layer(x) + return x + + + +class Encoder_Arch(nn.Module): + def __init__(self, + l_dim = 128, + h_dim = 384, + sep_num_layer = 2, + fusion_num_layer = 4, + patch_size = (2,4,4), + in_channel = 3 + ): + super(Encoder_Arch, self).__init__() + + self.is_first_chunk = True + self.tile_inference = False + + self.in_channel = in_channel + + self._build_linear_patch(in_channel=in_channel, out_channel_low=l_dim, out_channel_high=h_dim, pt=patch_size[0], ph=patch_size[1], pw=patch_size[2]) + + self.low_layer = ResNAF(num_layer=sep_num_layer, dim=l_dim) + self.high_layer = ResNAF(num_layer=sep_num_layer, dim=h_dim) + self.fusion_layer = ResNAF(num_layer=fusion_num_layer, dim=l_dim + h_dim) + + def _build_linear_patch(self, in_channel = 3, out_channel_low = 128, out_channel_high = 384, pt = 2, ph = 4, pw = 4): + patch_config = { + 'video_low': (pt, ph, pw), + 'video_high': (pt, ph, pw), + 'image_low': (1, ph, pw), + 'image_high': (1, ph, pw) + } + + for name, (t, h, w) in patch_config.items(): + if 'low' in name: + in_dim = in_channel * t * h * w + out_dim = out_channel_low + else: + out_dim = out_channel_high + in_dim = in_channel * t * h * w * 7 if 'video' in name else in_channel * t * h * w * 3 + proj = nn.Sequential( + Rearrange('b c (nt pt) (nh ph) (nw pw) -> b nt nh nw (c pt ph pw)', pt=t, ph=h, pw=w), + nn.Linear(in_dim, out_dim) + ) + self.add_module(f"{name}_proj", proj) + + + def _linear_patch(self, x, proj_type): + low_comp, high_comp = x[:, :self.in_channel], x[:, self.in_channel:] + return getattr(self, f"{proj_type}_low_proj")(low_comp), getattr(self, f"{proj_type}_high_proj")(high_comp) + + def _feature_transform(self, low_x, high_x): + low_x = self.low_layer(low_x) + high_x = self.high_layer(high_x) + x = torch.cat([low_x, high_x], dim=-1) + x = self.fusion_layer(x) + return x + + def forward(self, x): + xi, xv = x + xi_low, xi_high = self._linear_patch(x=xi, proj_type='image') + xv_low, xv_high = self._linear_patch(x=xv, proj_type='video') + + low_x = torch.cat([xi_low, xv_low], dim=1) + high_x = torch.cat([xi_high, xv_high], dim=1) + + return self._feature_transform(low_x=low_x, high_x=high_x) + + + + def encode(self, x): + xi, xv = x + if xi is not None and xv is not None: + xi_low, xi_high = self._linear_patch(x=xi, proj_type='image') + xv_low, xv_high = self._linear_patch(x=xv, proj_type='video') + + low_x = torch.cat([xi_low, xv_low], dim=1) + high_x = torch.cat([xi_high, xv_high], dim=1) + elif xi is not None: + low_x, high_x = self._linear_patch(x=xi, proj_type='image') + elif xv is not None: + low_x, high_x = self._linear_patch(x=xv, proj_type='video') + + return self._feature_transform(low_x=low_x, high_x=high_x) + + + +class Decoder_Arch(nn.Module): + def __init__(self, + l_dim = 128, + h_dim = 384, + sep_num_layer = 2, + fusion_num_layer = 4, + patch_size = (2,4,4), + in_channel = 3 + ): + super(Decoder_Arch, self).__init__() + + self.l_dim = l_dim + self.is_first_chunk = True + self.tile_inference = False + + self._build_linear_unpatch(in_channel=in_channel, out_channel_low=l_dim, out_channel_high=h_dim, pt=patch_size[0], ph=patch_size[1], pw=patch_size[2]) + + self.low_layer = ResNAF(num_layer=sep_num_layer, dim=l_dim) + self.high_layer = ResNAF(num_layer=sep_num_layer, dim=h_dim) + self.fusion_layer = ResNAF(num_layer=fusion_num_layer, dim=l_dim + h_dim) + + + def _build_linear_unpatch(self, in_channel = 3, out_channel_low = 128, out_channel_high = 384, pt = 2, ph = 4, pw = 4): + patch_config = { + 'video_low': (pt, ph, pw), + 'video_high': (pt, ph, pw), + 'image_low': (1, ph, pw), + 'image_high': (1, ph, pw) + } + + for name, (t, h, w) in patch_config.items(): + if 'low' in name: + out_dim = in_channel * t * h * w + in_dim = out_channel_low + else: + in_dim = out_channel_high + out_dim = in_channel * t * h * w * 7 if 'video' in name else in_channel * t * h * w * 3 + proj = nn.Sequential( + nn.Linear(in_dim, out_dim), + Rearrange('b nt nh nw (c pt ph pw) -> b c (nt pt) (nh ph) (nw pw)', pt=t, ph=h, pw=w), + ) + self.add_module(f"{name}_proj", proj) + + def _linear_unpatch(self, x, proj_type): + low_comp, high_comp = getattr(self, f"{proj_type}_low_proj")(x[0]), getattr(self, f"{proj_type}_high_proj")(x[1]) + return torch.cat([low_comp, high_comp], dim=1) + + def _feature_transform(self, x): + x = self.fusion_layer(x) + low_x = self.low_layer(x[:,:,:,:,:self.l_dim]) + high_x = self.high_layer(x[:,:,:,:,self.l_dim:]) + + return low_x, high_x + + + def decode(self, x, is_image = False): + low_x, high_x = self._feature_transform(x) + + if is_image: + xi = self._linear_unpatch(x=(low_x, high_x), proj_type='image') + return (xi, None) + + else: + if self.tile_inference and not self.is_first_chunk: + xv = self._linear_unpatch(x=(low_x, high_x), proj_type='video') + return (None, xv) + else: + xi = self._linear_unpatch(x=(low_x[:, :1], high_x[:, :1]), proj_type='image') + xv = self._linear_unpatch(x=(low_x[:, 1:], high_x[:, 1:]), proj_type='video') + return (xi, xv) + + def forward(self, x): + low_x, high_x = self._feature_transform(x) + xi = self._linear_unpatch(x=(low_x[:, :1], high_x[:, :1]), proj_type='image') + xv = self._linear_unpatch(x=(low_x[:, 1:], high_x[:, 1:]), proj_type='video') + return (xi, xv) + +class ISTA(nn.Module): + def __init__(self, + points_num = 512, + out_num = 4, + iter_num = 2, + layer_num = 2, + ): + super(ISTA, self).__init__() + phi_init = np.random.normal(0.0, (1 / points_num) ** 0.5, size=(out_num, points_num)) + self.phi = nn.Parameter(torch.from_numpy(phi_init).float(), requires_grad=True) + self.Q = nn.Parameter(torch.from_numpy(np.transpose(phi_init)).float(), requires_grad=True) + self.iter_num = iter_num + self.forward_l = nn.ModuleList() + self.backward_l = nn.ModuleList() + + for _ in range(self.iter_num): + self.forward_l.append(NAF(num_layer=layer_num, dim=points_num)) + self.backward_l.append(NAF(num_layer=layer_num, dim=points_num)) + + self.weights = nn.ParameterList() + self.etas = nn.ParameterList() + self.threshold = nn.ParameterList() + + for _ in range(self.iter_num): + self.threshold.append(nn.Parameter(torch.Tensor([0.01]), requires_grad=True)) + self.weights.append(nn.Parameter(torch.tensor(1.), requires_grad=True)) + + def sample(self, x): + b, t, h, w, d = x.shape + y = x.view(-1, d) @ self.phi.T + return y.view(b, t, h, w, -1) + + def recon(self, y): + b, t, h, w, c = y.shape + y = y.reshape(-1, c) + recon = torch.mm(y, self.Q.t()) + _, d = recon.shape + for i in range(self.iter_num): + recon_r = recon - self.weights[i] * torch.mm((torch.mm(recon, self.phi.t()) - y), self.phi) + recon = recon_r.reshape(b, t, h, w, -1) + recon = self.forward_l[i](recon) + recon = torch.mul(torch.sign(recon), F.relu(torch.abs(recon) - self.threshold[i])) + + recon = self.backward_l[i](recon).view(-1, d) + recon = recon_r + recon + return recon.view(b, t, h, w, -1) + + + def forward(self, x): + y = self.sample(x) + recon = self.recon(y) + return recon + + diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/cache/vgg.pth b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/cache/vgg.pth new file mode 100644 index 0000000000000000000000000000000000000000..f57dcf5cc764d61c8a460365847fb2137ff0a62d --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/cache/vgg.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a78928a0af1e5f0fcb1f3b9e8f8c3a2a5a3de244d830ad5c1feddc79b8432868 +size 7289 diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/discriminator.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/discriminator.py new file mode 100644 index 0000000000000000000000000000000000000000..b5b3ee43db5b456649578cc8554cd6993ec3152e --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/discriminator.py @@ -0,0 +1,130 @@ +from typing import Any, Union +import torch +import torch.nn as nn +import torch.nn.functional as F +from einops import rearrange +import functools + +class ActNorm(nn.Module): + def __init__(self, num_features, logdet=False, affine=True, allow_reverse_init=False): + assert affine + super().__init__() + self.logdet = logdet + self.loc = nn.Parameter(torch.zeros(1, num_features, 1, 1)) + self.scale = nn.Parameter(torch.ones(1, num_features, 1, 1)) + self.allow_reverse_init = allow_reverse_init + + self.register_buffer("initialized", torch.tensor(0, dtype=torch.uint8)) + + def initialize(self, input): + with torch.no_grad(): + flatten = input.permute(1, 0, 2, 3).contiguous().view(input.shape[1], -1) + mean = flatten.mean(1).unsqueeze(1).unsqueeze(2).unsqueeze(3).permute(1, 0, 2, 3) + std = flatten.std(1).unsqueeze(1).unsqueeze(2).unsqueeze(3).permute(1, 0, 2, 3) + + self.loc.data.copy_(-mean) + self.scale.data.copy_(1 / (std + 1e-6)) + + def forward(self, input, reverse=False): + if reverse: + return self.reverse(input) + if len(input.shape) == 2: + input = input[:, :, None, None] + squeeze = True + else: + squeeze = False + + _, _, height, width = input.shape + + if self.training and self.initialized.item() == 0: + self.initialize(input) + self.initialized.fill_(1) + + h = self.scale * (input + self.loc) + + if squeeze: + h = h.squeeze(-1).squeeze(-1) + + if self.logdet: + log_abs = torch.log(torch.abs(self.scale)) + logdet = height * width * torch.sum(log_abs) + logdet = logdet * torch.ones(input.shape[0]).to(input) + return h, logdet + + return h + + def reverse(self, output): + if self.training and self.initialized.item() == 0: + if not self.allow_reverse_init: + raise RuntimeError( + "Initializing ActNorm in reverse direction is " + "disabled by default. Use allow_reverse_init=True to enable." + ) + else: + self.initialize(output) + self.initialized.fill_(1) + + if len(output.shape) == 2: + output = output[:, :, None, None] + squeeze = True + else: + squeeze = False + + h = output / self.scale - self.loc + + if squeeze: + h = h.squeeze(-1).squeeze(-1) + return h + + + +class NLayerDiscriminator(nn.Module): + """Defines a PatchGAN discriminator as in Pix2Pix.""" + # https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/blob/master/models/networks.py + def __init__(self, input_nc=3, ndf=64, n_layers=3, use_actnorm=False): + """Construct a PatchGAN discriminator + Parameters: + input_nc (int) -- the number of channels in input images + ndf (int) -- the number of filters in the last conv layer + n_layers (int) -- the number of conv layers in the discriminator + """ + super(NLayerDiscriminator, self).__init__() + if not use_actnorm: + norm_layer = nn.BatchNorm2d + else: + norm_layer = ActNorm + if type(norm_layer) == functools.partial: # no need to use bias as BatchNorm2d has affine parameters + use_bias = norm_layer.func != nn.BatchNorm2d + else: + use_bias = norm_layer != nn.BatchNorm2d + + kw = 4 + padw = 1 + sequence = [nn.Conv2d(input_nc, ndf, kernel_size=kw, stride=2, padding=padw), nn.LeakyReLU(0.2, True)] + nf_mult = 1 + nf_mult_prev = 1 + for n in range(1, n_layers): # gradually increase the number of filters + nf_mult_prev = nf_mult + nf_mult = min(2**n, 8) + sequence += [ + nn.Conv2d(ndf * nf_mult_prev, ndf * nf_mult, kernel_size=kw, stride=2, padding=padw, bias=use_bias), + norm_layer(ndf * nf_mult), + nn.LeakyReLU(0.2, True), + ] + + nf_mult_prev = nf_mult + nf_mult = min(2**n_layers, 8) + sequence += [ + nn.Conv2d(ndf * nf_mult_prev, ndf * nf_mult, kernel_size=kw, stride=1, padding=padw, bias=use_bias), + norm_layer(ndf * nf_mult), + nn.LeakyReLU(0.2, True), + ] + + sequence += [ + nn.Conv2d(ndf * nf_mult, 1, kernel_size=kw, stride=1, padding=padw) + ] # output 1 channel prediction map + self.main = nn.Sequential(*sequence) + + def forward(self, input): + """Standard forward.""" + return self.main(input) \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/lpips.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/lpips.py new file mode 100644 index 0000000000000000000000000000000000000000..c63164f1804b37aa9e9a6739efb0b96138d4fb1b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/lpips.py @@ -0,0 +1,230 @@ +"""Stripped version of https://github.com/richzhang/PerceptualSimilarity/tree/master/models""" + +import os, hashlib +import requests +from tqdm import tqdm + +import torch +import torch.nn as nn +from torchvision import models +from collections import namedtuple +import torchvision +URL_MAP = { + "vgg_lpips": "https://heibox.uni-heidelberg.de/f/607503859c864bc1b30b/?dl=1" +} + +CKPT_MAP = { + "vgg_lpips": "vgg.pth" +} + +MD5_MAP = { + "vgg_lpips": "d507d7349b931f0638a25a48a722f98a" +} + +def download(url, local_path, chunk_size=1024): + os.makedirs(os.path.split(local_path)[0], exist_ok=True) + with requests.get(url, stream=True) as r: + total_size = int(r.headers.get("content-length", 0)) + with tqdm(total=total_size, unit="B", unit_scale=True) as pbar: + with open(local_path, "wb") as f: + for data in r.iter_content(chunk_size=chunk_size): + if data: + f.write(data) + pbar.update(chunk_size) + + +def md5_hash(path): + with open(path, "rb") as f: + content = f.read() + return hashlib.md5(content).hexdigest() + + +def get_ckpt_path(name, root, check=False): + assert name in URL_MAP + path = os.path.join(root, CKPT_MAP[name]) + if not os.path.exists(path) or (check and not md5_hash(path) == MD5_MAP[name]): + print("Downloading {} model from {} to {}".format(name, URL_MAP[name], path)) + download(URL_MAP[name], path) + md5 = md5_hash(path) + assert md5 == MD5_MAP[name], md5 + return path + + +class LPIPS(nn.Module): + # Learned perceptual metric + def __init__(self, use_dropout=True): + super().__init__() + self.scaling_layer = ScalingLayer() + self.chns = [64, 128, 256, 512, 512] # vg16 features + self.net = vgg16(pretrained=True, requires_grad=False) + self.lin0 = NetLinLayer(self.chns[0], use_dropout=use_dropout) + self.lin1 = NetLinLayer(self.chns[1], use_dropout=use_dropout) + self.lin2 = NetLinLayer(self.chns[2], use_dropout=use_dropout) + self.lin3 = NetLinLayer(self.chns[3], use_dropout=use_dropout) + self.lin4 = NetLinLayer(self.chns[4], use_dropout=use_dropout) + self.load_from_pretrained() + for param in self.parameters(): + param.requires_grad = False + + def load_from_pretrained(self, name="vgg_lpips"): + ckpt = get_ckpt_path(name, os.path.join(os.path.dirname(os.path.abspath(__file__)), "cache")) + self.load_state_dict(torch.load(ckpt, map_location=torch.device("cpu")), strict=False) + print("loaded pretrained LPIPS loss from {}".format(ckpt)) + + @classmethod + def from_pretrained(cls, name="vgg_lpips"): + if name is not "vgg_lpips": + raise NotImplementedError + model = cls() + ckpt = get_ckpt_path(name, os.path.join(os.path.dirname(os.path.abspath(__file__)), "cache")) + r = model.load_state_dict(torch.load(ckpt, map_location=torch.device("cpu")), strict=False) + print(r) + return model + + def forward(self, input, target): + in0_input, in1_input = (self.scaling_layer(input), self.scaling_layer(target)) + outs0, outs1 = self.net(in0_input), self.net(in1_input) + feats0, feats1, diffs = {}, {}, {} + lins = [self.lin0, self.lin1, self.lin2, self.lin3, self.lin4] + for kk in range(len(self.chns)): + feats0[kk], feats1[kk] = normalize_tensor(outs0[kk]), normalize_tensor(outs1[kk]) + diffs[kk] = (feats0[kk] - feats1[kk]) ** 2 + + res = [spatial_average(lins[kk].model(diffs[kk]), keepdim=True) for kk in range(len(self.chns))] + val = res[0] + for l in range(1, len(self.chns)): + # print(res[l].shape) + val += res[l] + + return val + + + + + +class ScalingLayer(nn.Module): + def __init__(self): + super(ScalingLayer, self).__init__() + self.register_buffer('shift', torch.Tensor([-.030, -.088, -.188])[None, :, None, None]) + self.register_buffer('scale', torch.Tensor([.458, .448, .450])[None, :, None, None]) + + def forward(self, inp): + return (inp - self.shift) / self.scale + + +class NetLinLayer(nn.Module): + """ A single linear layer which does a 1x1 conv """ + def __init__(self, chn_in, chn_out=1, use_dropout=False): + super(NetLinLayer, self).__init__() + layers = [nn.Dropout(), ] if (use_dropout) else [] + layers += [nn.Conv2d(chn_in, chn_out, 1, stride=1, padding=0, bias=False), ] + self.model = nn.Sequential(*layers) + + +class vgg16(torch.nn.Module): + def __init__(self, requires_grad=False, pretrained=True): + super(vgg16, self).__init__() + vgg_pretrained_features = models.vgg16(pretrained=pretrained).features + self.slice1 = torch.nn.Sequential() + self.slice2 = torch.nn.Sequential() + self.slice3 = torch.nn.Sequential() + self.slice4 = torch.nn.Sequential() + self.slice5 = torch.nn.Sequential() + self.N_slices = 5 + for x in range(4): + self.slice1.add_module(str(x), vgg_pretrained_features[x]) + for x in range(4, 9): + self.slice2.add_module(str(x), vgg_pretrained_features[x]) + for x in range(9, 16): + self.slice3.add_module(str(x), vgg_pretrained_features[x]) + for x in range(16, 23): + self.slice4.add_module(str(x), vgg_pretrained_features[x]) + for x in range(23, 30): + self.slice5.add_module(str(x), vgg_pretrained_features[x]) + if not requires_grad: + for param in self.parameters(): + param.requires_grad = False + + def forward(self, X): + h = self.slice1(X) + h_relu1_2 = h + h = self.slice2(h) + h_relu2_2 = h + h = self.slice3(h) + h_relu3_3 = h + h = self.slice4(h) + h_relu4_3 = h + h = self.slice5(h) + h_relu5_3 = h + vgg_outputs = namedtuple("VggOutputs", ['relu1_2', 'relu2_2', 'relu3_3', 'relu4_3', 'relu5_3']) + out = vgg_outputs(h_relu1_2, h_relu2_2, h_relu3_3, h_relu4_3, h_relu5_3) + return out + + +def normalize_tensor(x,eps=1e-10): + norm_factor = torch.sqrt(torch.sum(x**2,dim=1,keepdim=True)) + return x/(norm_factor+eps) + + +def spatial_average(x, keepdim=True): + return x.mean([2,3],keepdim=keepdim) + + +class ResNetLPIPS(nn.Module): + # Learned perceptual metric + def __init__(self, use_dropout=True): + super().__init__() + net, _ = clip.load(device='cpu', name='RN50') + self.net = net.visual + self.net.attnpool = nn.Identity() + for param in self.parameters(): + param.requires_grad = False + + def forward(self, input, target): + + outs0, outs1 = self.net(input), self.net(target) + #feats0, feats1= normalize_tensor(outs0), normalize_tensor(outs1) + diffs = ((outs0 - outs1) ** 2 ) #(feats0 - feats1) ** 2 + + return diffs + + +class MeanShift(nn.Conv2d): + def __init__(self, data_mean, data_std, data_range=1, norm=True): + c = len(data_mean) + super(MeanShift, self).__init__(c, c, kernel_size=1) + std = torch.Tensor(data_std) + self.weight.data = torch.eye(c).view(c, c, 1, 1) + if norm: + self.weight.data.div_(std.view(c, 1, 1, 1)) + self.bias.data = -1 * data_range * torch.Tensor(data_mean) + self.bias.data.div_(std) + else: + self.weight.data.mul_(std.view(c, 1, 1, 1)) + self.bias.data = data_range * torch.Tensor(data_mean) + self.requires_grad = False + +class VGGPerceptualLoss(torch.nn.Module): + def __init__(self, rank): + super(VGGPerceptualLoss, self).__init__() + blocks = [] + pretrained = True + self.vgg_pretrained_features = models.vgg19(pretrained=pretrained).features + self.normalize = MeanShift([0.485, 0.456, 0.406], [0.229, 0.224, 0.225], norm=True).to(rank) + for param in self.parameters(): + param.requires_grad = False + + def forward(self, Y, X, indices=None): + X = self.normalize(X) + Y = self.normalize(Y) + indices = [2, 7, 12, 21, 30] + weights = [1.0/2.6, 1.0/4.8, 1.0/3.7, 1.0/5.6, 10/1.5] + k = 0 + loss = 0 + for i in range(indices[-1]): + X = self.vgg_pretrained_features[i](X) + Y = self.vgg_pretrained_features[i](Y) + if (i+1) in indices: + loss += weights[k] * (X - Y.detach()).abs().mean() * 0.1 + k += 1 + return loss \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/vae.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/vae.py new file mode 100644 index 0000000000000000000000000000000000000000..7acc228efac8a8b318fff172d0cefc8fe686a93a --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/modules/vae.py @@ -0,0 +1,73 @@ +import torch +import numpy as np + +class DiagonalGaussianDistribution(object): + def __init__(self, parameters, deterministic=False): + self.parameters = parameters + self.mean, self.logvar = parameters #torch.chunk(parameters, 2, dim=1) + self.logvar = torch.clamp(self.logvar, -30.0, 20.0) + self.deterministic = deterministic + self.std = torch.exp(0.5 * self.logvar) + self.var = torch.exp(self.logvar) + if self.deterministic: + self.var = self.std = torch.zeros_like(self.mean).to(device=self.mean.device) + + def sample(self): + x = self.mean + self.std * torch.randn(self.mean.shape).to(device=self.mean.device) + return x + + def kl(self, other=None): + if self.deterministic: + return torch.Tensor([0.]) + else: + if other is None: + return 0.5 * torch.sum(torch.pow(self.mean, 2) + + self.var - 1.0 - self.logvar, + dim=[1, 2, 3]) + else: + return 0.5 * torch.sum( + torch.pow(self.mean - other.mean, 2) / other.var + + self.var / other.var - 1.0 - self.logvar + other.logvar, + dim=[1, 2, 3]) + + def nll(self, sample, dims=[1,2,3]): + if self.deterministic: + return torch.Tensor([0.]) + logtwopi = np.log(2.0 * np.pi) + return 0.5 * torch.sum( + logtwopi + self.logvar + torch.pow(sample - self.mean, 2) / self.var, + dim=dims) + + def mode(self): + return self.mean + + + +def normal_kl(mean1, logvar1, mean2, logvar2): + """ + source: https://github.com/openai/guided-diffusion/blob/27c20a8fab9cb472df5d6bdd6c8d11c8f430b924/guided_diffusion/losses.py#L12 + Compute the KL divergence between two gaussians. + Shapes are automatically broadcasted, so batches can be compared to + scalars, among other use cases. + """ + tensor = None + for obj in (mean1, logvar1, mean2, logvar2): + if isinstance(obj, torch.Tensor): + tensor = obj + break + assert tensor is not None, "at least one argument must be a Tensor" + + # Force variances to be Tensors. Broadcasting helps convert scalars to + # Tensors, but it does not work for torch.exp(). + logvar1, logvar2 = [ + x if isinstance(x, torch.Tensor) else torch.tensor(x).to(tensor) + for x in (logvar1, logvar2) + ] + + return 0.5 * ( + -1.0 + + logvar2 + - logvar1 + + torch.exp(logvar1 - logvar2) + + ((mean1 - mean2) ** 2) * torch.exp(-logvar2) + ) \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__init__.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/__init__.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..57ee8867d464f096fe8391638dd7e4bcb0833ec2 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/__init__.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/__init__.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/__init__.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..184a742bbb796a7bdd68af44a3c1ae075c736b90 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/__init__.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/__init__.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/__init__.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..b07eb56eb4837e2f72764729a0175b0e0ff5b7f1 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/__init__.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/gan_loss.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/gan_loss.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..86ae2261d5733f2b772298b5615709578ca40010 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/gan_loss.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/gan_loss.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/gan_loss.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..82011565780bb1bff31f01de8b0539b8e6b80ba0 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/gan_loss.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/gan_loss.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/gan_loss.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..8efb4b047b74b2c3a711ab65e2abb1fc2e0dd6fb Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/gan_loss.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/patcher_utils.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/patcher_utils.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..7f34d48a5a72bb19293f2fdeca167ef7c570335a Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/patcher_utils.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/patcher_utils.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/patcher_utils.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..bb9ebe7ac4df24296156c7db38193058e58770b2 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/patcher_utils.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/patcher_utils.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/patcher_utils.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..18aa41670f08e2d55fdb9aa751a4aa14cd8695cd Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/patcher_utils.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/video_utils.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/video_utils.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..f0627719bde6ef77680c75f737c95db7cfeaa8fc Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/__pycache__/video_utils.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/callbacks.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/callbacks.py new file mode 100644 index 0000000000000000000000000000000000000000..7e7d4fa62a683e079a3e5b41f3644bd836ee9f19 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/callbacks.py @@ -0,0 +1,120 @@ +import os +import numpy as np +from PIL import Image + +import torch +import torchvision +from pytorch_lightning.callbacks import Callback +from pytorch_lightning.utilities.distributed import rank_zero_only + +import random +from .utils import save_video_grid + + + +class VideoLogger(Callback): + def __init__(self, batch_frequency, max_videos, clamp=True, increase_log_steps=True): + super().__init__() + self.batch_freq = batch_frequency + self.max_videos = max_videos + self.log_steps = [2 ** n for n in range(int(np.log2(self.batch_freq)) + 1)] + if not increase_log_steps: + self.log_steps = [self.batch_freq] + self.clamp = clamp + + + @rank_zero_only + def log_local(self, save_dir, split, videos, + global_step, current_epoch, batch_idx): + root = os.path.join(save_dir, "videos", split) + for k in videos: + grid = videos[k] + 0.5 + filename = "gs-{:06}_e-{:06}_b-{:06}_{}.mp4".format( + global_step, + current_epoch, + batch_idx, + k) + path = os.path.join(root, filename) + os.makedirs(os.path.split(path)[0], exist_ok=True) + save_video_grid(grid, path) + + def log_vid(self, pl_module, batch, batch_idx, split="train"): + # print(batch_idx, self.batch_freq, self.check_frequency(batch_idx) and hasattr(pl_module, "log_videos") and callable(pl_module.log_videos) and self.max_videos > 0) + if (self.check_frequency(batch_idx) and # batch_idx % self.batch_freq == 0 + hasattr(pl_module, "log_videos") and + callable(pl_module.log_videos) and + self.max_videos > 0): + # print(batch_idx, self.batch_freq, self.check_frequency(batch_idx)) + logger = type(pl_module.logger) + + is_train = pl_module.training + if is_train: + pl_module.eval() + + with torch.no_grad(): + videos = pl_module.log_videos(batch, split=split, batch_idx=batch_idx) + + for k in videos: + N = min(videos[k].shape[0], self.max_videos) + videos[k] = videos[k][:N] + if isinstance(videos[k], torch.Tensor): + videos[k] = videos[k].detach().cpu() + if self.clamp: + videos[k] = torch.clamp(videos[k], -0.5, 0.5) + + self.log_local(pl_module.logger.save_dir, split, videos, + pl_module.global_step, pl_module.current_epoch, batch_idx) + + if is_train: + pl_module.train() + + def check_frequency(self, batch_idx): + if (batch_idx % self.batch_freq) == 0 or (batch_idx in self.log_steps): + try: + self.log_steps.pop(0) + except IndexError: + pass + return True + return False + + def on_train_batch_end(self, trainer, pl_module, outputs, batch, batch_idx, dataloader_idx): + if batch[0]['video'].ndim == 4: + return + self.log_vid(pl_module, batch, batch_idx, split="train") + + def on_validation_batch_end(self, trainer, pl_module, outputs, batch, batch_idx, dataloader_idx): + self.log_vid(pl_module, batch, batch_idx, split="val") + + +class DatasetCallback(Callback): + def __init__(self, initial_batch_size, new_batch_size, step_threshold): + self.initial_batch_size = initial_batch_size + self.new_batch_size = new_batch_size + self.step_threshold = step_threshold + + def on_train_batch_start(self, trainer, pl_module, batch, batch_idx, dataloader_idx): + if trainer.global_step == self.step_threshold: + # 更新 DataLoader 的 batch_size + trainer.train_dataloader = trainer.video_data._dataloader(train=True, batch_size=self.new_batch_size) # self.new_batch_size + print(f'Batch size changed to {self.new_batch_size} at step {self.step_threshold}') + # def __init__(self): + # self.seqlen_list = [17, 21, 17, 25, 21, 29, 33, 17, 21, 17] + +# #临时取消 + +# def on_batch_start(self, trainer, pl_module): +# seq_len = random.randint(0, 9) +# trainer.train_dataloader.dataset.datasets[0].sequence_length = self.seqlen_list[seq_len] + +# def on_train_batch_start(self, trainer, pl_module, outputs, batch, batch_idx, dataloader_idx): +# seq_len = batch_idx % 10 +# trainer.train_dataloader.dataset.datasets[dataloader_idx].sequence_length = self.seqlen_list[seq_len] +# return +# #为啥上面那个没报错 下边那个报错说是list呢?? +# def on_epoch_end(self, trainer, pl_module): +# with open('tmp_shape2.txt', 'a') as f: +# print(trainer.current_epoch, file=f ) +# #trainer.train_dataloader[0].sampler.set_epoch(trainer.current_epoch) +# return + + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/gan_loss.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/gan_loss.py new file mode 100644 index 0000000000000000000000000000000000000000..471c1f56ec17164ef871239460b8b600c4146425 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/gan_loss.py @@ -0,0 +1,147 @@ +from typing import Any, Union +import torch +import torch.nn as nn +import torch.nn.functional as F +from einops import rearrange +import functools +from ..modules.discriminator import NLayerDiscriminator + +def hinge_d_loss(logits_real, logits_fake): + loss_real = torch.mean(F.relu(1.0 - logits_real)) + loss_fake = torch.mean(F.relu(1.0 + logits_fake)) + d_loss = 0.5 * (loss_real + loss_fake) + return d_loss + + +def vanilla_d_loss(logits_real, logits_fake): + d_loss = 0.5 * (torch.mean(F.softplus(-logits_real)) + torch.mean(F.softplus(logits_fake))) + return d_loss + + +def adopt_weight(weight, global_step, threshold=0, value=0.0): + if global_step < threshold: + weight = value + return weight + + +def _sigmoid_cross_entropy_with_logits(labels, logits): + """ + non-saturating loss + """ + zeros = torch.zeros_like(logits, dtype=logits.dtype) + condition = logits >= zeros + relu_logits = torch.where(condition, logits, zeros) + neg_abs_logits = torch.where(condition, -logits, logits) + return relu_logits - logits * labels + torch.log1p(torch.exp(neg_abs_logits)) + + +def non_saturate_gen_loss(logits_fake): + """ + logits_fake: [B 1 H W] + """ + B = logits_fake.shape[0] + logits_fake = logits_fake.reshape(B, -1) + logits_fake = torch.mean(logits_fake, dim=-1) + gen_loss = torch.mean(_sigmoid_cross_entropy_with_logits(labels=torch.ones_like(logits_fake), logits=logits_fake)) + return gen_loss + + +def lecam_reg(real_pred, fake_pred, lecam_ema): + reg = torch.mean(F.relu(real_pred - lecam_ema.logits_fake_ema).pow(2)) + torch.mean( + F.relu(lecam_ema.logits_real_ema - fake_pred).pow(2) + ) + return reg + + +class LeCAM_EMA(object): + # https://github.com/TencentARC/SEED-Voken/blob/main/src/Open_MAGVIT2/modules/losses/vqperceptual.py + def __init__(self, init=0.0, decay=0.999): + self.logits_real_ema = init + self.logits_fake_ema = init + self.decay = decay + + def update(self, logits_real, logits_fake): + self.logits_real_ema = self.logits_real_ema * self.decay + torch.mean(logits_real).item() * (1 - self.decay) + self.logits_fake_ema = self.logits_fake_ema * self.decay + torch.mean(logits_fake).item() * (1 - self.decay) + +def weights_init(m): + classname = m.__class__.__name__ + if classname.find("Conv") != -1: + nn.init.normal_(m.weight.data, 0.0, 0.02) + elif classname.find("BatchNorm") != -1: + nn.init.normal_(m.weight.data, 1.0, 0.02) + nn.init.constant_(m.bias.data, 0) + +class AdversarialLoss(nn.Module): + def __init__( + self, + disc_num_layers: int = 3, + disc_in_channels: int = 3, + disc_weight: float = 0.2, + lecam_loss_weight: float = 0.005, + disc_loss: str = "hinge", + dims: int = 3, + gen_loss_cross_entropy: bool = True, + ): + super().__init__() + self.dims = dims + assert disc_loss in ["hinge", "vanilla"] + self.discriminator = NLayerDiscriminator( + input_nc=disc_in_channels, n_layers=disc_num_layers, use_actnorm=False + ).apply(weights_init) + + self.disc_loss = hinge_d_loss if disc_loss == "hinge" else vanilla_d_loss + self.discriminator_weight = disc_weight + + self.gen_loss_cross_entropy = gen_loss_cross_entropy + self.lecam_loss_weight = lecam_loss_weight + if self.lecam_loss_weight > 0: + self.lecam_ema = LeCAM_EMA() + + def get_trainable_parameters(self) -> Any: + return self.discriminator.parameters() + + def forward( + self, + inputs, + reconstructions, + optimizer_idx, + ): + + if optimizer_idx == 0: + if self.dims > 2: + inputs, reconstructions = map( + lambda x: rearrange(x, "b c t h w -> (b t) c h w"), + (inputs, reconstructions), + ) + + # generator update + logits_fake = self.discriminator(reconstructions) + + if not self.gen_loss_cross_entropy: + g_loss = -torch.mean(logits_fake) + else: + g_loss = non_saturate_gen_loss(logits_fake) + + g_loss = self.discriminator_weight * g_loss + return g_loss + + + if optimizer_idx == 1: + inputs, reconstructions = map( + lambda x: rearrange(x, "b c t h w -> (b t) c h w"), + (inputs, reconstructions), + ) + + logits_real = self.discriminator(inputs.contiguous().detach()) + logits_fake = self.discriminator(reconstructions.contiguous().detach()) + + non_saturate_d_loss = self.disc_loss(logits_real, logits_fake) + + if self.lecam_loss_weight > 0: + self.lecam_ema.update(logits_real, logits_fake) + lecam_loss = lecam_reg(logits_real, logits_fake, self.lecam_ema) + d_loss = lecam_loss * self.lecam_loss_weight + non_saturate_d_loss + else: + d_loss = non_saturate_d_loss + return d_loss diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/patcher_utils.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/patcher_utils.py new file mode 100644 index 0000000000000000000000000000000000000000..acefc5dd7001eaefac5be490b2d5037270d8338a --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/patcher_utils.py @@ -0,0 +1,246 @@ +import os +import random +import shutil +import sys +from datetime import datetime +import math +import numpy as np +import pywt +import torch +from torch.nn import Module +import numpy as np +import torch.nn as nn +import torch.nn.functional as F +import torch.nn as nn + +_PERSISTENT = False + + +class Patcher(torch.nn.Module): + def __init__(self, rescale = True): + super().__init__() + self.register_buffer( + "wavelets", torch.tensor([0.7071067811865476, 0.7071067811865476]), persistent=_PERSISTENT + ) + self.register_buffer( + "_arange", + torch.arange(2), + persistent=_PERSISTENT, + ) + + self.rescale = rescale + for param in self.parameters(): + param.requires_grad = False + + + + def _2ddwt(self, x, mode="reflect", rescale=False): + dtype = x.dtype + h = self.wavelets + x = x.squeeze(2) + + n = h.shape[0] + g = x.shape[1] + hl = h.flip(0).reshape(1, 1, -1).repeat(g, 1, 1) + hh = (h * ((-1) ** self._arange)).reshape(1, 1, -1).repeat(g, 1, 1) + hh = hh.to(dtype=dtype) + hl = hl.to(dtype=dtype) + + x = F.pad(x, pad=(n - 2, n - 1, n - 2, n - 1), mode=mode).to(dtype) + xl = F.conv2d(x, hl.unsqueeze(2), groups=g, stride=(1, 2)) + xh = F.conv2d(x, hh.unsqueeze(2), groups=g, stride=(1, 2)) + xll = F.conv2d(xl, hl.unsqueeze(3), groups=g, stride=(2, 1)) + xlh = F.conv2d(xl, hh.unsqueeze(3), groups=g, stride=(2, 1)) + xhl = F.conv2d(xh, hl.unsqueeze(3), groups=g, stride=(2, 1)) + xhh = F.conv2d(xh, hh.unsqueeze(3), groups=g, stride=(2, 1)) + + out = torch.cat([xll, xlh, xhl, xhh], dim=1) + if rescale: + out = out * 2 + return out.unsqueeze(2) + + def _3ddwt(self, x, mode="reflect", rescale=False): + dtype = x.dtype + h = self.wavelets + + n = h.shape[0] + g = x.shape[1] + hl = h.flip(0).reshape(1, 1, -1).repeat(g, 1, 1) + hh = (h * ((-1) ** self._arange)).reshape(1, 1, -1).repeat(g, 1, 1) + hh = hh.to(dtype=dtype) + hl = hl.to(dtype=dtype) + + # Handles temporal axis. + x = F.pad( + x, pad=(max(0, n - 2), n - 1, n - 2, n - 1, n - 2, n - 1), mode=mode + ).to(dtype) + xl = F.conv3d(x, hl.unsqueeze(3).unsqueeze(4), groups=g, stride=(2, 1, 1)) + xh = F.conv3d(x, hh.unsqueeze(3).unsqueeze(4), groups=g, stride=(2, 1, 1)) + + # Handles spatial axes. + xll = F.conv3d(xl, hl.unsqueeze(2).unsqueeze(4), groups=g, stride=(1, 2, 1)) + xlh = F.conv3d(xl, hh.unsqueeze(2).unsqueeze(4), groups=g, stride=(1, 2, 1)) + xhl = F.conv3d(xh, hl.unsqueeze(2).unsqueeze(4), groups=g, stride=(1, 2, 1)) + xhh = F.conv3d(xh, hh.unsqueeze(2).unsqueeze(4), groups=g, stride=(1, 2, 1)) + + xlll = F.conv3d(xll, hl.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2)) + xllh = F.conv3d(xll, hh.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2)) + xlhl = F.conv3d(xlh, hl.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2)) + xlhh = F.conv3d(xlh, hh.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2)) + xhll = F.conv3d(xhl, hl.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2)) + xhlh = F.conv3d(xhl, hh.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2)) + xhhl = F.conv3d(xhh, hl.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2)) + xhhh = F.conv3d(xhh, hh.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2)) + + out = torch.cat([xlll, xllh, xlhl, xlhh, xhll, xhlh, xhhl, xhhh], dim=1) + if rescale: + out = out * (2 * torch.sqrt(torch.tensor(2.0))) + return out + + def forward(self, x): + if x.shape[2] > 1: + if x.shape[2] % 2 == 1: + xi, xv = torch.split(x, [1, x.shape[2] - 1], dim=2) + xi = self._2ddwt(xi, rescale=self.rescale) + xv = self._3ddwt(xv, rescale=self.rescale) + return xi, xv + else: + xv = self._3ddwt(x, rescale=self.rescale) + return None, xv + + return (self._2ddwt(x, rescale=self.rescale), None) + + +class UnPatcher(torch.nn.Module): + + def __init__(self, rescale = True): + super().__init__() + self.register_buffer( + "wavelets", torch.tensor([0.7071067811865476, 0.7071067811865476]), persistent=_PERSISTENT + ) + self.register_buffer( + "_arange", + torch.arange(2), + persistent=_PERSISTENT, + ) + self.rescale = rescale + for param in self.parameters(): + param.requires_grad = False + + def forward(self, x): + xi, xv = x + if xi is not None and xv is not None: + xi = self._2didwt(xi, rescale=self.rescale) + xv = self._3didwt(xv, rescale=self.rescale) + return torch.cat([xi.unsqueeze(2), xv], dim=2) + elif xv is None and xi is not None: + return self._2didwt(xi, rescale=self.rescale) + elif xv is not None and xi is None: + return self._3didwt(xv, rescale=self.rescale) + + def _2didwt(self, x, mode="reflect", rescale=False): + dtype = x.dtype + h = self.wavelets + n = h.shape[0] + x = x.squeeze(2) + + g = x.shape[1] // 4 + hl = h.flip([0]).reshape(1, 1, -1).repeat([g, 1, 1]) + hh = (h * ((-1) ** self._arange)).reshape(1, 1, -1).repeat(g, 1, 1) + hh = hh.to(dtype=dtype) + hl = hl.to(dtype=dtype) + + xll, xlh, xhl, xhh = torch.chunk(x.to(dtype), 4, dim=1) + + # Inverse transform. + yl = torch.nn.functional.conv_transpose2d( + xll, hl.unsqueeze(3), groups=g, stride=(2, 1), padding=(n - 2, 0) + ) + yl += torch.nn.functional.conv_transpose2d( + xlh, hh.unsqueeze(3), groups=g, stride=(2, 1), padding=(n - 2, 0) + ) + yh = torch.nn.functional.conv_transpose2d( + xhl, hl.unsqueeze(3), groups=g, stride=(2, 1), padding=(n - 2, 0) + ) + yh += torch.nn.functional.conv_transpose2d( + xhh, hh.unsqueeze(3), groups=g, stride=(2, 1), padding=(n - 2, 0) + ) + y = torch.nn.functional.conv_transpose2d( + yl, hl.unsqueeze(2), groups=g, stride=(1, 2), padding=(0, n - 2) + ) + y += torch.nn.functional.conv_transpose2d( + yh, hh.unsqueeze(2), groups=g, stride=(1, 2), padding=(0, n - 2) + ) + + if rescale: + y = y / 2 + return y + + def _3didwt(self, x, mode="reflect", rescale=False): + dtype = x.dtype + h = self.wavelets + n = h.shape[0] + + g = x.shape[1] // 8 # split into 8 spatio-temporal filtered tesnors. + hl = h.flip([0]).reshape(1, 1, -1).repeat([g, 1, 1]) + hh = (h * ((-1) ** self._arange)).reshape(1, 1, -1).repeat(g, 1, 1) + hl = hl.to(dtype=dtype) + hh = hh.to(dtype=dtype) + + xlll, xllh, xlhl, xlhh, xhll, xhlh, xhhl, xhhh = torch.chunk(x, 8, dim=1) + + # Height height transposed convolutions. + xll = F.conv_transpose3d( + xlll, hl.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2) + ) + xll += F.conv_transpose3d( + xllh, hh.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2) + ) + + xlh = F.conv_transpose3d( + xlhl, hl.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2) + ) + xlh += F.conv_transpose3d( + xlhh, hh.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2) + ) + + xhl = F.conv_transpose3d( + xhll, hl.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2) + ) + xhl += F.conv_transpose3d( + xhlh, hh.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2) + ) + + xhh = F.conv_transpose3d( + xhhl, hl.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2) + ) + xhh += F.conv_transpose3d( + xhhh, hh.unsqueeze(2).unsqueeze(3), groups=g, stride=(1, 1, 2) + ) + + # Handles width transposed convolutions. + xl = F.conv_transpose3d( + xll, hl.unsqueeze(2).unsqueeze(4), groups=g, stride=(1, 2, 1) + ) + xl += F.conv_transpose3d( + xlh, hh.unsqueeze(2).unsqueeze(4), groups=g, stride=(1, 2, 1) + ) + xh = F.conv_transpose3d( + xhl, hl.unsqueeze(2).unsqueeze(4), groups=g, stride=(1, 2, 1) + ) + xh += F.conv_transpose3d( + xhh, hh.unsqueeze(2).unsqueeze(4), groups=g, stride=(1, 2, 1) + ) + + # Handles time axis transposed convolutions. + x = F.conv_transpose3d( + xl, hl.unsqueeze(3).unsqueeze(4), groups=g, stride=(2, 1, 1) + ) + x += F.conv_transpose3d( + xh, hh.unsqueeze(3).unsqueeze(4), groups=g, stride=(2, 1, 1) + ) + + if rescale: + x = x / (2 * torch.sqrt(torch.tensor(2.0))) + return x + diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/utils.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/utils.py new file mode 100644 index 0000000000000000000000000000000000000000..647af2a3d1a6e0507fcefd89ebf6dd3e9d8edb8b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/utils.py @@ -0,0 +1,171 @@ +import torch +import imageio + +import math +import numpy as np +import skvideo.io + +import sys +import pdb as pdb_original + +class ForkedPdb(pdb_original.Pdb): + """A Pdb subclass that may be used + from a forked multiprocessing child + + """ + def interaction(self, *args, **kwargs): + _stdin = sys.stdin + try: + sys.stdin = open('/dev/stdin') + pdb_original.Pdb.interaction(self, *args, **kwargs) + finally: + sys.stdin = _stdin + + + +# Shifts src_tf dim to dest dim +# i.e. shift_dim(x, 1, -1) would be (b, c, t, h, w) -> (b, t, h, w, c) +def shift_dim(x, src_dim=-1, dest_dim=-1, make_contiguous=True): + n_dims = len(x.shape) + if src_dim < 0: + src_dim = n_dims + src_dim + if dest_dim < 0: + dest_dim = n_dims + dest_dim + + assert 0 <= src_dim < n_dims and 0 <= dest_dim < n_dims + + dims = list(range(n_dims)) + del dims[src_dim] + + permutation = [] + ctr = 0 + for i in range(n_dims): + if i == dest_dim: + permutation.append(src_dim) + else: + permutation.append(dims[ctr]) + ctr += 1 + x = x.permute(permutation) + if make_contiguous: + x = x.contiguous() + return x + + +# reshapes tensor start from dim i (inclusive) +# to dim j (exclusive) to the desired shape +# e.g. if x.shape = (b, thw, c) then +# view_range(x, 1, 2, (t, h, w)) returns +# x of shape (b, t, h, w, c) +def view_range(x, i, j, shape): + shape = tuple(shape) + + n_dims = len(x.shape) + if i < 0: + i = n_dims + i + + if j is None: + j = n_dims + elif j < 0: + j = n_dims + j + + assert 0 <= i < j <= n_dims + + x_shape = x.shape + target_shape = x_shape[:i] + shape + x_shape[j:] + return x.view(target_shape) + + +def accuracy(output, target, topk=(1,)): + """Computes the accuracy over the k top predictions for the specified values of k""" + with torch.no_grad(): + maxk = max(topk) + batch_size = target.size(0) + + _, pred = output.topk(maxk, 1, True, True) + pred = pred.t() + correct = pred.eq(target.reshape(1, -1).expand_as(pred)) + + res = [] + for k in topk: + correct_k = correct[:k].reshape(-1).float().sum(0, keepdim=True) + res.append(correct_k.mul_(100.0 / batch_size)) + return res + + +def tensor_slice(x, begin, size): + assert all([b >= 0 for b in begin]) + size = [l - b if s == -1 else s + for s, b, l in zip(size, begin, x.shape)] + assert all([s >= 0 for s in size]) + + slices = [slice(b, b + s) for b, s in zip(begin, size)] + return x[slices] + + +def adopt_weight(global_step, threshold=0, value=0.): + weight = 1 + if global_step < threshold: + weight = value + return weight + + +def save_video_grid(video, fname, nrow=None, fps=3): + b, c, t, h, w = video.shape + video = video.permute(0, 2, 3, 4, 1).contiguous() + + video = (video.detach().cpu().numpy() * 255).astype('uint8') + if nrow is None: + nrow = math.ceil(math.sqrt(b)) + ncol = math.ceil(b / nrow) + padding = 0 #临时修改 + video_grid = np.zeros((t, (padding + h) * nrow + padding, + (padding + w) * ncol + padding, c), dtype='uint8') + # print(video_grid.shape) + for i in range(b): + r = i // ncol + c = i % ncol + start_r = (padding + h) * r + start_c = (padding + w) * c + video_grid[:, start_r:start_r + h, start_c:start_c + w] = video[i] + video = [] + for i in range(t): + video.append(video_grid[i]) + imageio.mimsave(fname, video, fps=fps) + # skvideo.io.vwrite(fname, video_grid, inputdict={'-r': '5'}) + # print('saved videos to', fname) + + +def comp_getattr(args, attr_name, default=None): + if hasattr(args, attr_name): + return getattr(args, attr_name) + else: + return default + + +def visualize_tensors(t, name=None, nest=0): + if name is not None: + print(name, "current nest: ", nest) + print("type: ", type(t)) + if 'dict' in str(type(t)): + print(t.keys()) + for k in t.keys(): + if t[k] is None: + print(k, "None") + else: + if 'Tensor' in str(type(t[k])): + print(k, t[k].shape) + elif 'dict' in str(type(t[k])): + print(k, 'dict') + visualize_tensors(t[k], name, nest + 1) + elif 'list' in str(type(t[k])): + print(k, len(t[k])) + visualize_tensors(t[k], name, nest + 1) + elif 'list' in str(type(t)): + print("list length: ", len(t)) + for t2 in t: + visualize_tensors(t2, name, nest + 1) + elif 'Tensor' in str(type(t)): + print(t.shape) + else: + print(t) + return "" diff --git a/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/video_utils.py b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/video_utils.py new file mode 100644 index 0000000000000000000000000000000000000000..2153e91852e00d3e4f3536e5a8403497ba999a6f --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/LeanVAE/utils/video_utils.py @@ -0,0 +1,824 @@ +from email.policy import default + +import numbers +import random +import re +from enum import Enum + +import numpy as np +from PIL import Image +import torch +import torchvision as tv +import torchvision.transforms as transforms +import torch.nn.functional as F +from decord import VideoReader + +from torch.nn.functional import interpolate as img_tensor_resize +from torch.nn.functional import pad as img_tensor_pad +from torch.nn.modules.utils import _quadruple +from torchvision.transforms.functional import pad as img_pad +from torchvision.transforms.functional import resize as img_resize +class LMDB_Image: + def __init__(self, image, id): + self.channels = image.shape[2] + self.size = image.shape[:2] + self.image = image.tobytes() + self.id = id + + def get_image(self): + image = np.frombuffer(self.image, dtype=np.uint8) + return image.reshape(*self.size, self.channels) + +# save each video as a class with byte data, which can be decoded from lmdb database. +class LMDB_VIDEO: + def __init__(self, image, id): + self.size = image.shape + self.image = image.tobytes() + self.id = id + + def get_image(self): + image = np.frombuffer(self.image, dtype=np.uint8) + return image.reshape(*self.size) +_pil_interpolation_to_str = { + Image.NEAREST: "PIL.Image.NEAREST", + Image.BILINEAR: "PIL.Image.BILINEAR", + Image.BICUBIC: "PIL.Image.BICUBIC", + Image.LANCZOS: "PIL.Image.LANCZOS", + Image.HAMMING: "PIL.Image.HAMMING", + Image.BOX: "PIL.Image.BOX", +} + + +class VideoNorm(object): + """Apply Normalization to Image Pixels on GPU""" + + def __init__( + self, + mean=[0.5, 0.5, 0.5], + std=[1.0, 1.0, 1.0], + #mean=[0.48145466, 0.4578275, 0.40821073], + #std=[0.26862954, 0.26130258, 0.27577711], + ): + # self.mean = torch.tensor(mean).cuda().view(1, 3, 1, 1) + # self.std = torch.tensor(std).cuda().view(1, 3, 1, 1) + self.mean = torch.tensor(mean).view(1, 3, 1, 1) + self.std = torch.tensor(std).view(1, 3, 1, 1) + + def __call__(self, img): + """ + Args: + img: float image tensors, (N, 3, H, W) + Returns: + img: normalized float image tensors + """ + if torch.max(img) > 1 and self.mean.max() <= 1: + img.div_(255.0) + re = img.sub_(self.mean).div_(self.std) + return re + + + + +class VideoResizeSquare(object): + def __init__(self, out_size, interpolation="nearest"): + assert isinstance(out_size, int) + self.out_size = out_size + self.interpolation = interpolation + + def __call__(self, video): + """ + Args: + img (torch.tensor): video to be scaled. + + Returns: + torch.tensor: Rescaled video. + """ + if isinstance(video, torch.Tensor): + if len(video.shape) == 4: + t, h, w, c = video.shape + assert ( + c == 3 + ), "Expecting 3-channel color video, got video of shape {}".format( + video.shape + ) + else: + raise RuntimeError( + "Expecting 4-dimensional tensor of shape (b,t,h,w), got {}".format( + video.shape + ) + ) + + # t, h, w, c -> t, c, h, w + video = video.permute(0, 3, 1, 2) + short_side = h if h < w else w + resized_video = img_tensor_resize( + video, + size=((self.out_size, self.out_size)), + mode=self.interpolation, + ) + + # t, c, h, w -> t, h, w, c + return resized_video.permute(0, 2, 3, 1) + + else: + # in other data class, the order of shape might be different. + raise NotImplementedError( + "Support only torch.Tensor as input, got {}".format(type(video)) + ) + + def __repr__(self): + return self.__class__.__name__ + "(size={0}, interpolation={1})".format( + self.out_size, self.interpolation + ) + + + +def load_video_from_path_tvio( + video_path, + frm_sampling_strategy, + height=None, + width=None, + fps=-1, + num_frm=None, +): + video = tv.io.read_video(rf"{video_path}", pts_unit="sec") + if not height or not width: + sampled_frms = np.array(video[0]) + else: + # T, H, W, C + sampled_frms_tensor = video[0] + # expected: t, c, h, w + resize_func = VideoResizeSquare(out_size=height) + sampled_frms_tensor = resize_func(sampled_frms_tensor) + sampled_frms = np.array(sampled_frms_tensor) + + specified_num_frm = num_frm + default_fps = video[2]["video_fps"] + vlen = sampled_frms.shape[0] + + if fps != -1: + # resample the video to the specified fps + duration = vlen / default_fps + num_frames_to_sample = int(duration * fps) + resample_indices = np.linspace( + 0, vlen - 1, num_frames_to_sample + ).astype(int) + + # print(default_fps, fps, resample_indices) + sampled_frms = sampled_frms[resample_indices] + default_fps = fps + + vlen = sampled_frms.shape[0] + if num_frm is None: + num_frm = vlen + + num_frm = min(num_frm, vlen) + + if frm_sampling_strategy == "uniform": + frame_indices = np.linspace(0, vlen - 1, num_frm).astype(int) + + elif frm_sampling_strategy == "rand": + # frame_indices = sorted(random.sample(range(vlen), num_frm)) + rand_start = random.randint(0, vlen - num_frm) + frame_indices = np.array(range(rand_start, rand_start + num_frm)).astype(int) + + elif frm_sampling_strategy == "center": + center = vlen // 2 + if num_frm % 2 ==0: + frame_indices = np.array(range(center - num_frm // 2, center + num_frm // 2)).astype(int) + else: + frame_indices = np.array(range(center - num_frm // 2, center + num_frm // 2 + 1)).astype(int) + + elif frm_sampling_strategy == "all": + frame_indices = np.arange(0, vlen).astype(int) + + else: + raise NotImplementedError( + "Invalid sampling strategy {} ".format(frm_sampling_strategy) + ) + + raw_sample_frms = sampled_frms[ + frame_indices + ] + + if specified_num_frm is None: + masks = np.ones(len(raw_sample_frms), dtype=np.uint8) + + # pad the video if the number of frames is less than specified + elif len(raw_sample_frms) < specified_num_frm: + prev_length = len(raw_sample_frms) + zeros = np.zeros( + (specified_num_frm - prev_length, height, width, 3), + dtype=np.uint8, + ) + raw_sample_frms = np.concatenate((raw_sample_frms, zeros), axis=0) + masks = np.zeros(specified_num_frm, dtype=np.uint8) + masks[:prev_length] = 1 + + else: + masks = np.ones(specified_num_frm, dtype=np.uint8) + + + return raw_sample_frms, masks + + +def load_video_from_path_decord( + video_path, + frm_sampling_strategy, + height=None, + width=None, + start_time=None, + end_time=None, + fps=-1, + num_frm=None, +): + #return np.zeros((num_frm, 256, 256, 3), dtype=np.uint8), np.ones(num_frm, dtype=np.uint8) + specified_num_frm = num_frm + + if not height or not width: + vr = VideoReader(rf"{video_path}") + else: + vr = VideoReader(video_path, width=width, height=height) + + default_fps = vr.get_avg_fps() + if default_fps <= fps: + fps = -1 + + if fps != -1: + # resample the video to the specified fps + duration = len(vr) / default_fps + num_frames_to_sample = int(duration * fps) + resample_indices = np.linspace( + 0, len(vr) - 1, num_frames_to_sample + ).astype(int) + + # print(default_fps, fps, resample_indices) + sampled_frms = vr.get_batch(resample_indices).asnumpy().astype(np.uint8) + default_fps = fps + + + else: + sampled_frms = vr.get_batch(np.arange(0, len(vr), 1, dtype=int)).asnumpy().astype(np.uint8) + + vlen = sampled_frms.shape[0] + + if num_frm is None: + num_frm = vlen + + num_frm = min(num_frm, vlen) + + if start_time or end_time: + assert ( + fps > 0 + ), "must provide video fps if specifying start and end time." + start_idx = min(int(start_time * fps), vlen) + end_idx = min(int(end_time * fps), vlen) + + else: + start_idx, end_idx = 0, vlen + + if frm_sampling_strategy == "uniform": + frame_indices = np.linspace(0, vlen - 1, num_frm).astype(int) + + elif frm_sampling_strategy == "nlvl_uniform": + frame_indices = np.arange( + start_idx, end_idx, vlen / num_frm + ).astype(int) + + elif frm_sampling_strategy == "nlvl_rand": + frame_indices = np.arange( + start_idx, end_idx, vlen / num_frm + ).astype(int) + + strides = [ + frame_indices[i] - frame_indices[i - 1] + for i in range(1, len(frame_indices)) + ] + [vlen - frame_indices[-1]] + pertube = np.array( + [np.random.randint(0, stride) for stride in strides] + ) + + frame_indices = frame_indices + pertube + + elif frm_sampling_strategy == "rand": + # frame_indices = sorted(random.sample(range(vlen), num_frm)) + rand_start = random.randint(0, vlen - num_frm) + frame_indices = np.array(range(rand_start, rand_start + num_frm)).astype(int) + + elif frm_sampling_strategy == "center": + center = vlen // 2 + if num_frm % 2 ==0: + frame_indices = np.array(range(center - num_frm // 2, center + num_frm // 2)).astype(int) + else: + frame_indices = np.array(range(center - num_frm // 2, center + num_frm // 2 + 1)).astype(int) + + elif frm_sampling_strategy == "headtail": + frame_indices_head = sorted( + random.sample(range(vlen // 2), num_frm // 2) + ) + frame_indices_tail = sorted( + random.sample(range(vlen // 2, vlen), num_frm // 2) + ) + frame_indices = frame_indices_head + frame_indices_tail + + elif frm_sampling_strategy == "all": + frame_indices = np.arange(0, vlen).astype(int) + + + elif frm_sampling_strategy == "rand_sep": + sep = random.choice([1,2,3,4,5,6,7,8]) + while (sep * num_frm) > vlen: + sep = random.choice([1,2,3,4,5,6,7,8]) + rand_start = random.randint(0, vlen - sep * num_frm) + frame_indices = np.array(range(rand_start, rand_start + sep * num_frm, sep)).astype(int) + + else: + raise NotImplementedError( + "Invalid sampling strategy {} ".format(frm_sampling_strategy) + ) + + raw_sample_frms = sampled_frms[ + frame_indices + ] + + if specified_num_frm is None: + masks = np.ones(len(raw_sample_frms), dtype=np.uint8) + + # pad the video if the number of frames is less than specified + elif len(raw_sample_frms) < specified_num_frm: + prev_length = len(raw_sample_frms) + zeros = np.zeros( + (specified_num_frm - prev_length, height, width, 3), + dtype=np.uint8, + ) + raw_sample_frms = np.concatenate((raw_sample_frms, zeros), axis=0) + masks = np.zeros(specified_num_frm, dtype=np.uint8) + masks[:prev_length] = 1 + + else: + masks = np.ones(specified_num_frm, dtype=np.uint8) + + return raw_sample_frms, masks + + +def load_video_from_path_lmdb( + sampled_frms, + frm_sampling_strategy, + height=None, + width=None, + start_time=None, + end_time=None, + fps=-1, + num_frm=None, +): + #return np.zeros((num_frm, 256, 256, 3), dtype=np.uint8), np.ones(num_frm, dtype=np.uint8) + specified_num_frm = num_frm + + vlen = sampled_frms.shape[0] + + if num_frm is None: + num_frm = vlen + + num_frm = min(num_frm, vlen) + + if start_time or end_time: + assert ( + fps > 0 + ), "must provide video fps if specifying start and end time." + start_idx = min(int(start_time * fps), vlen) + end_idx = min(int(end_time * fps), vlen) + + else: + start_idx, end_idx = 0, vlen + + if frm_sampling_strategy == "uniform": + frame_indices = np.linspace(0, vlen - 1, num_frm).astype(int) + + elif frm_sampling_strategy == "nlvl_uniform": + frame_indices = np.arange( + start_idx, end_idx, vlen / num_frm + ).astype(int) + + elif frm_sampling_strategy == "nlvl_rand": + frame_indices = np.arange( + start_idx, end_idx, vlen / num_frm + ).astype(int) + + strides = [ + frame_indices[i] - frame_indices[i - 1] + for i in range(1, len(frame_indices)) + ] + [vlen - frame_indices[-1]] + pertube = np.array( + [np.random.randint(0, stride) for stride in strides] + ) + + frame_indices = frame_indices + pertube + + elif frm_sampling_strategy == "rand": + # frame_indices = sorted(random.sample(range(vlen), num_frm)) + rand_start = random.randint(0, vlen - num_frm) + frame_indices = np.array(range(rand_start, rand_start + num_frm)).astype(int) + + elif frm_sampling_strategy == "center": + center = vlen // 2 + if num_frm % 2 ==0: + frame_indices = np.array(range(center - num_frm // 2, center + num_frm // 2)).astype(int) + else: + frame_indices = np.array(range(center - num_frm // 2, center + num_frm // 2 + 1)).astype(int) + + elif frm_sampling_strategy == "headtail": + frame_indices_head = sorted( + random.sample(range(vlen // 2), num_frm // 2) + ) + frame_indices_tail = sorted( + random.sample(range(vlen // 2, vlen), num_frm // 2) + ) + frame_indices = frame_indices_head + frame_indices_tail + + elif frm_sampling_strategy == "all": + frame_indices = np.arange(0, vlen).astype(int) + + + elif frm_sampling_strategy == "rand_sep": + sep = random.choice([1,2,3,4,5,6,7,8]) + while (sep * num_frm) > vlen: + sep = random.choice([1,2,3,4,5,6,7,8]) + rand_start = random.randint(0, vlen - sep * num_frm) + frame_indices = np.array(range(rand_start, rand_start + sep * num_frm, sep)).astype(int) + + else: + raise NotImplementedError( + "Invalid sampling strategy {} ".format(frm_sampling_strategy) + ) + + raw_sample_frms = sampled_frms[ + frame_indices + ] + + if specified_num_frm is None: + masks = np.ones(len(raw_sample_frms), dtype=np.uint8) + + # pad the video if the number of frames is less than specified + elif len(raw_sample_frms) < specified_num_frm: + prev_length = len(raw_sample_frms) + zeros = np.zeros( + (specified_num_frm - prev_length, height, width, 3), + dtype=np.uint8, + ) + raw_sample_frms = np.concatenate((raw_sample_frms, zeros), axis=0) + masks = np.zeros(specified_num_frm, dtype=np.uint8) + masks[:prev_length] = 1 + + else: + masks = np.ones(specified_num_frm, dtype=np.uint8) + + return raw_sample_frms, masks + +def image_to_tensor(image: np.ndarray, keepdim: bool = True) -> torch.Tensor: + """Converts a numpy image to a PyTorch 4d tensor image. + Args: + image (numpy.ndarray): image of the form :math:`(H, W, C)`, :math:`(H, W)` or + :math:`(B, H, W, C)`. + keepdim (bool): If ``False`` unsqueeze the input image to match the shape + :math:`(B, H, W, C)`. Default: ``True`` + Returns: + torch.Tensor: tensor of the form :math:`(B, C, H, W)` if keepdim is ``False``, + :math:`(C, H, W)` otherwise. + """ + if not isinstance(image, (np.ndarray,)): + raise TypeError( + "Input type must be a numpy.ndarray. Got {}".format(type(image)) + ) + + if len(image.shape) > 4 or len(image.shape) < 2: + raise ValueError( + "Input size must be a two, three or four dimensional array" + ) + + input_shape = image.shape + tensor: torch.Tensor = torch.from_numpy(image) + + if len(input_shape) == 2: + # (H, W) -> (1, H, W) + tensor = tensor.unsqueeze(0) + elif len(input_shape) == 3: + # (H, W, C) -> (C, H, W) + tensor = tensor.permute(2, 0, 1) + elif len(input_shape) == 4: + # (B, H, W, C) -> (B, C, H, W) + tensor = tensor.permute(0, 3, 1, 2) + keepdim = True # no need to unsqueeze + else: + raise ValueError( + "Cannot process image with shape {}".format(input_shape) + ) + + return tensor.unsqueeze(0) if not keepdim else tensor + + +def get_padding(image, max_w, max_h, pad_all=False): + # keep the images to upper-left corner + if isinstance(image, torch.Tensor): + h, w = image.shape[-2:] + else: + w, h = image.size + h_padding, v_padding = max_w - w, max_h - h + if pad_all: + h_padding /= 2 + v_padding /= 2 + l_pad = h_padding if h_padding % 1 == 0 else h_padding + 0.5 + t_pad = v_padding if v_padding % 1 == 0 else v_padding + 0.5 + r_pad = h_padding if h_padding % 1 == 0 else h_padding - 0.5 + b_pad = v_padding if v_padding % 1 == 0 else v_padding - 0.5 + else: + l_pad, t_pad = 0, 0 + r_pad, b_pad = h_padding, v_padding + if isinstance(image, torch.Tensor): + padding = (int(l_pad), int(r_pad), int(t_pad), int(b_pad)) + else: + padding = (int(l_pad), int(t_pad), int(r_pad), int(b_pad)) + return padding + + +class ImagePad(object): + def __init__(self, max_w, max_h, fill=0, padding_mode="constant"): + assert isinstance(fill, (numbers.Number, str, tuple)) + assert padding_mode in ["constant", "edge", "reflect", "symmetric"] + self.max_w = max_w + self.max_h = max_h + self.fill = fill + self.padding_mode = padding_mode + + def __call__(self, img): + """ + Args: + img (PIL Image): Image to be padded. + + Returns: + PIL Image: Padded image. + """ + if isinstance(img, torch.Tensor): + paddings = _quadruple(get_padding(img, self.max_w, self.max_h)) + return img_tensor_pad(img, paddings, self.padding_mode, self.fill) + return img_pad( + img, + get_padding(img, self.max_w, self.max_h), + self.fill, + self.padding_mode, + ) + + def __repr__(self): + return ( + self.__class__.__name__ + + "(padding={0}, fill={1}, padding_mode={2})".format( + self.fill, self.padding_mode + ) + ) + + +def get_resize_size(image, max_size): + """ + Args: + image: PIL Image or torch.tensor + max_size: + + Returns: + + Note the height/width order difference + >>> pil_img = Image.open("raw_img_tensor.jpg") + >>> pil_img.size + (640, 480) # (width, height) + >>> np_img = np.array(pil_img) + >>> np_img.shape + (480, 640, 3) # (height, width, 3) + """ + # note the order of height and width for different inputs + if isinstance(image, torch.Tensor): + # width, height = image.shape[-2:] + height, width = image.shape[-2:] + else: + width, height = image.size + + if height >= width: + ratio = width * 1.0 / height + new_height = max_size + new_width = new_height * ratio + else: + ratio = height * 1.0 / width + new_width = max_size + new_height = new_width * ratio + size = (int(new_height), int(new_width)) + return size + + +class VideoRandomSquareCrop(object): + def __init__(self, crop_size, p=0.5): + assert isinstance(crop_size, int) + self.crop_size = crop_size + self.p = p + + def __call__(self, video): + """ + Args: + img (torch.tensor): video to be cropped. + + Returns: + torch.tensor: cropped video. + """ + if isinstance(video, torch.Tensor): + if len(video.shape) == 4: + b, t, h, w = video.shape + else: + raise RuntimeError( + "Expecting 4-dimensional tensor of shape (b,t,h,w), got {}".format( + video.shape + ) + ) + + if random.uniform(0, 1) < self.p: + video = torch.flip(video, (3,)) + + x = random.randint(0, h - self.crop_size) + y = random.randint(0, w - self.crop_size) + + return video[:, :, x : x + self.crop_size, y : y + self.crop_size] + + else: + if random.uniform(0, 1) < self.p: + video = np.ascontiguousarray(np.flip(video, (2,))) + t, h, w, c = video.shape + x = random.randint(0, h - self.crop_size) + y = random.randint(0, w - self.crop_size) + + return video[:, x : x + self.crop_size, y : y + self.crop_size, :] + + +class ImageResize(object): + """Resize the input image (torch.tensor) to the given size. + + Args: + max_size (int): Desired output size. If size is a sequence like + (h, w), output size will be matched to this. If size is an int, + smaller edge of the image will be matched to this number. + i.e, if height > width, then image will be rescaled to + (size * height / width, size) + interpolation (int, optional): Desired interpolation. Default is + ``PIL.Image.BILINEAR`` + """ + + def __init__(self, max_size, interpolation=Image.BILINEAR): + assert isinstance(max_size, int) + self.max_size = max_size + self.interpolation = interpolation + + def __call__(self, img): + """ + Args: + img (torch.tensor): Image to be scaled. + + Returns: + torch.tensor: Rescaled image. + """ + if isinstance(img, torch.Tensor): + assert isinstance(self.interpolation, str) + return img_tensor_resize( + img, + size=get_resize_size(img, self.max_size), + mode=self.interpolation, + align_corners=False, + ) + return img_resize( + img, get_resize_size(img, self.max_size), self.interpolation + ) + + def __repr__(self): + interpolate_str = _pil_interpolation_to_str[self.interpolation] + return self.__class__.__name__ + "(size={0}, interpolation={1})".format( + self.size, interpolate_str + ) + + +def get_imagenet_transform(min_size=600, max_size=1000): + """parameters from https://github.com/pytorch/examples/blob/master/imagenet/main.py + This simply crop the center square from the image + """ + if min_size != 600: + import warnings + + warnings.warn( + f"Warning: min_size is not used in image transform, " + f"setting min_size will have no effect." + ) + return transforms.Compose( + [ + ImageResize( + max_size, Image.BILINEAR + ), # longer side will be resized to 1000 + ImagePad(max_size, max_size), # pad to 1000 * 1000 + ] + ) + + +class ImageNorm(object): + """Apply Normalization to Image Pixels on GPU""" + + def __init__(self, mean, std): + self.mean = torch.tensor(mean).cuda().view(1, 1, 3, 1, 1) + self.std = torch.tensor(std).cuda().view(1, 1, 3, 1, 1) + # assert max(std) <= 1 and min(std) >= 0\ + # or max(mean) <= 1 and min(mean) >= 0,\ + # "Please provide mean or std within range [0, 1]" + + def __call__(self, img): + """ + Args: + img: float image tensors, (B, N, 3, H, W) + + Returns: + img: normalized float image tensors + """ + if torch.max(img) > 1 and self.mean.max() <= 1: + img.div_(255.0) + return img.sub_(self.mean).div_(self.std) + + +def chunk_list(examples, chunk_size=2, pad_to_divisible=True): + """ + Args: + examples: iterable, examples grouped by image/video + chunk_size: int, number of examples in each chunk. + pad_to_divisible: bool, pad the examples to be divisible by chunk_size. + >>> test_examples = [3, 4, 5, 6, 7] + >>> chunk_list(test_examples, chunk_size=2, pad_to_divisible=True) + [[3, 4], [5, 6], [7, 7]] # the lst element has some randomness + >>> chunk_list(test_examples, chunk_size=2, pad_to_divisible=False) + [[3, 4], [5, 6], [7]] + """ + n_examples = len(examples) + remainder = n_examples % chunk_size + if pad_to_divisible and remainder > 0: + n_pad = chunk_size - remainder + pad = random.choices(examples, k=n_pad) # with replacement + examples = examples + pad + n_examples = len(examples) + remainder = 0 + chunked_examples = [] + n_chunks = int(n_examples / chunk_size) + n_chunks = n_chunks + 1 if remainder > 0 else n_chunks + for i in range(n_chunks): + chunked_examples.append(examples[i * chunk_size : (i + 1) * chunk_size]) + return chunked_examples + + +# def repeat_tensor_rows(raw_tensor, row_repeats): +# """ repeat raw_tensor[i] row_repeats[i] times. +# Args: +# raw_tensor: (B, *) +# row_repeats: list(int), len(row_repeats) == len(raw_tensor) +# """ +# assert len(raw_tensor) == len(raw_tensor), "Has to be the same length" +# if sum(row_repeats) == len(row_repeats): +# return raw_tensor +# else: +# indices = torch.LongTensor( +# flat_list_of_lists([[i] * r for i, r in enumerate(row_repeats)]) +# ).to(raw_tensor.device) +# return raw_tensor.index_select(0, indices) + + +def pre_caption(caption, max_words=50): + caption = re.sub( + r"([.!\"()*#:;~])", + " ", + caption.lower(), + ) + caption = re.sub( + r"\s{2,}", + " ", + caption, + ) + caption = caption.rstrip("\n") + caption = caption.strip(" ") + + # truncate caption + caption_words = caption.split(" ") + if len(caption_words) > max_words: + caption = " ".join(caption_words[:max_words]) + + return caption + + + + +class InterpolationMode(Enum): + """Interpolation modes + Available interpolation methods are ``nearest``, ``bilinear``, ``bicubic``, ``box``, ``hamming``, and ``lanczos``. + """ + + NEAREST = "nearest" + BILINEAR = "bilinear" + BICUBIC = "bicubic" + # For PIL compatibility + BOX = "box" + HAMMING = "hamming" + LANCZOS = "lanczos" \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/README.md b/Abnormal-CT-Generation-MultiDisease/README.md new file mode 100644 index 0000000000000000000000000000000000000000..b2db10e5c830399479fe0caeaf15b403690d648f --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/README.md @@ -0,0 +1,85 @@ +# Generative Enhancement for 3D Medical Images + +This is the official code for https://arxiv.org/abs/2403.12852. + + + +## Installation + +Clone this repository and install packages: +``` +git clone https://github.com/HKU-MedAI/GEM-3D.git +conda env create --file environment.yml +conda activate gem +``` + +## Preparation + +1. The preprocessed datasets can be found [here](https://connecthkuhk-my.sharepoint.com/:u:/g/personal/ltzhu99_connect_hku_hk/ES0_s1XN3_BDhQn6W3cPvPgB-LnX9SJqUcBQ3dA8g-jqZA?e=3qiz5v). (For BraTS, we use the version from [Medical Segmentation Decathlon](http://medicaldecathlon.com/#tasks) and only use the FLAIR modality. For abdomen dataset, we crop the outlier regions where contain too many empty slices. Note that we need the gt segmentation for benchmark, and we mannually split the training splits. The datasets are preprocessed with [nnUNet](https://github.com/MIC-DKFZ/nnUNet).) + +2. Download [KL-f8 AE](https://ommer-lab.com/files/latent-diffusion/kl-f8.zip) from [LDM](https://github.com/CompVis/latent-diffusion). + + +3. You can download our pretrained models [here](https://connecthkuhk-my.sharepoint.com/:u:/g/personal/ltzhu99_connect_hku_hk/EdGegBBAYVZMnhLhAvs46OUBRDPVmYBI6IqX20K3OWGTQA?e=McrYWl). + +## Training +``` +# stage 1, 150k iters +CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python main.py --base configs/latent-diffusion/brain_stage1.yaml -t --gpus 0,1,2,3,4,5,6,7, + +# Edit the path-to-stage1-ckpt keyword in the stage 2 yaml +vi configs/latent-diffusion/brain_stage2.yaml + +# stage 2, 50k iters +CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python main.py --base configs/latent-diffusion/brain_stage2.yaml -t --gpus 0,1,2,3,4,5,6,7, +``` +``` +# baseline (a modified version of Make-A-Volume[MICCAI 2023]): use the yaml with 'base' and train the models in two stages, the same as the commands above +``` +``` +# position conditioned slice generation model (one-stage), 150k iters +CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python main.py --base configs/latent-diffusion/brain_slice.yaml -t --gpus 0,1,2,3,4,5,6,7, +``` +Note that the training costs more than 30G for each card, requiring GPUs like V100 and A100. + +And the training can also be conducted with fewer cards. + +## Inference + +Prepare the models in the directory `infer_model/`. +``` +# postfix: +# {ic,icma,ig,igma} correspond to the 4 settings in the paper (Tab.1) +# test means that data comes from the test split (Tab.2) +# base corresponds to the baseline +# slice corresponds to position conditioned slice generation model +python inference_{dataset}_{postfix}.py + +## e.g., +python inference_brain_ic.py +``` +The inference costs GPU memory within 20G. + +## Citation + +If you find our work useful, please kindly cite as: +``` +@article{zhu2024generative, + title={Generative Enhancement for 3D Medical Images}, + author={Zhu, Lingting and Codella, Noel and Chen, Dongdong and Jin, Zhenchao and Yuan, Lu and Yu, Lequan}, + journal={arXiv preprint arXiv:2403.12852}, + year={2024} +} + +@inproceedings{zhu2023make, + title={Make-a-volume: Leveraging latent diffusion models for cross-modality 3d brain mri synthesis}, + author={Zhu, Lingting and Xue, Zeyue and Jin, Zhenchao and Liu, Xian and He, Jingzhen and Liu, Ziwei and Yu, Lequan}, + booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention}, + pages={592--601}, + year={2023}, + organization={Springer} +} +``` + +## Acknowledgement +The codebase is developed based on [SD](https://github.com/CompVis/stable-diffusion) (Rombach et al.). diff --git a/Abnormal-CT-Generation-MultiDisease/config.sh b/Abnormal-CT-Generation-MultiDisease/config.sh new file mode 100644 index 0000000000000000000000000000000000000000..9ddb3c334b5b1657f61fd1026d2c41c4d3691560 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/config.sh @@ -0,0 +1,34 @@ +# stage 1, 150k iters +CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python main.py --base configs/latent-diffusion/brain_stage1.yaml -t --gpus 0,1,2,3,4,5,6,7, + +# Edit the path-to-stage1-ckpt keyword in the stage 2 yaml +vi configs/latent-diffusion/brain_stage2.yaml + +# stage 2, 50k iters +CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python main.py --base configs/latent-diffusion/brain_stage2.yaml -t --gpus 0,1,2,3,4,5,6,7, + + + + +CUDA_VISIBLE_DEVICES=0 python main.py --base configs/latent-diffusion/brain_stage1.yaml -t --gpus 0, + +CUDA_VISIBLE_DEVICES=0 python main.py --base configs/latent-diffusion/brain_slice.yaml -t --gpus 0, + +CUDA_VISIBLE_DEVICES=0 python main.py --base configs/latent-diffusion/abdomen_3d.yaml -t --gpus 0, + +CUDA_VISIBLE_DEVICES=0 python main.py --base configs/latent-diffusion/brain_slice_atten.yaml -t --gpus 0, + +CUDA_VISIBLE_DEVICES=0 python main.py --base configs/latent-diffusion/brain_slice_atten2.yaml -t --gpus 0, + +CUDA_VISIBLE_DEVICES=0 python main.py --base configs/latent-diffusion/brain_slice_atten3.yaml -t --gpus 0, + +CUDA_VISIBLE_DEVICES=0 python main.py --base configs/latent-diffusion/brain_slice_atten4.yaml -t --gpus 0, + +CUDA_VISIBLE_DEVICES=0 python main.py --base configs/latent-diffusion/brain_slice_atten5.yaml -t --gpus 0, + + +CUDA_VISIBLE_DEVICES=0 python main.py --base configs/latent-diffusion/full_ct_3d_with_body_mask2.yaml -t --gpus 0, --resume ./logs/full_ct_3d_with_body_mask2 + +CUDA_VISIBLE_DEVICES=0 python main.py --base configs/latent-diffusion/full_ct_3d_with_body_mask_finetune.yaml -t --gpus 0, --resume ./logs/full_ct_3d_with_body_mask_finetune2 + +pip install beartype PyWavelets diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d.yaml new file mode 100644 index 0000000000000000000000000000000000000000..56952ba5c1861d34258b413f45dc69cc4c530eeb --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-09-13T02-58-56_brain_slice/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 100 + validation: + target: ldm.data.volume_dataset.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e7de493c0863442030a795d1184fa9727f1d8cba --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-10-30T18-29-30_brain_slice_13class_new_randomclass_alldata/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset_13class_randomclass.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset103_AbdomenAtlas2.0_base' + data_repeat: 4 + validation: + target: ldm.data.volume_dataset_13class_randomclass.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset103_AbdomenAtlas2.0_base' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..88c5c6da6a5b9b8d62ad8dd80ca9b98965ff1e93 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata2.yaml @@ -0,0 +1,94 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-10-31T18-08-08_brain_slice_13class_new_randomclass_alldata2/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset_13class_randomclass.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset103_AbdomenAtlas2.0_base' + data_repeat: 4 + data_file: 'data_split/splits_final_all.json' + validation: + target: ldm.data.volume_dataset_13class_randomclass.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset103_AbdomenAtlas2.0_base' + data_file: 'data_split/splits_final_all.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata3.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata3.yaml new file mode 100644 index 0000000000000000000000000000000000000000..15236399e4f11d469c47b99bf7c9931ab995584b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata3.yaml @@ -0,0 +1,94 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-11-04T07-42-05_brain_slice_13class_new_randomclass_alldata3/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset_13class_randomclass.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset104_AbdomenAtlas2.0_1102' + data_repeat: 4 + data_file: 'data_split/splits_final_all2.json' + validation: + target: ldm.data.volume_dataset_13class_randomclass.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset104_AbdomenAtlas2.0_1102' + data_file: 'data_split/splits_final_all2.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata4.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata4.yaml new file mode 100644 index 0000000000000000000000000000000000000000..04fc5db5434b658303f65569d44e0b911b29d63e --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_13class_randomclass_alldata4.yaml @@ -0,0 +1,94 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-11-06T12-22-22_brain_slice_13class_new_randomclass_alldata4/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset_13class_randomclass.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset105_AbdomenAtlas2.0_1106' + data_repeat: 4 + data_file: 'data_split/splits_final_all3.json' + validation: + target: ldm.data.volume_dataset_13class_randomclass.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset105_AbdomenAtlas2.0_1106' + data_file: 'data_split/splits_final_all3.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_28class_randomclass.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_28class_randomclass.yaml new file mode 100644 index 0000000000000000000000000000000000000000..8ad8ed99a640b36e035ecef611336963e505aa65 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_28class_randomclass.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-10-13T09-17-24_brain_slice_28class_new_randomclass/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset_28class_randomclass.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset101_AbdomenAtlasInternal' + data_repeat: 4 + validation: + target: ldm.data.volume_dataset_28class_randomclass.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset101_AbdomenAtlasInternal' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_28class_randomclass2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_28class_randomclass2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..aed19612e02b38d31704969a5c14515dae8d459d --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_28class_randomclass2.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-10-19T04-42-07_brain_slice_28class_new_randomclass/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset_28class_randomclass.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset101_AbdomenAtlasInternal' + data_repeat: 4 + validation: + target: ldm.data.volume_dataset_28class_randomclass.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset101_AbdomenAtlasInternal' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_28class_randomclass3.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_28class_randomclass3.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e60a5c4a29a67d0b43f867a1791e0b43e8e3fc2d --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_28class_randomclass3.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-10-17T08-03-29_brain_slice_28class_new_randomclass/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset_28class_randomclass.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset101_AbdomenAtlasInternal' + data_repeat: 4 + validation: + target: ldm.data.volume_dataset_28class_randomclass.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset101_AbdomenAtlasInternal' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_atten3.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_atten3.yaml new file mode 100644 index 0000000000000000000000000000000000000000..2675ecb7c09de350866fc702f5b5b1b05d8c0d6c --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_atten3.yaml @@ -0,0 +1,93 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: hybrid + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-10-11T04-22-28_brain_slice_atten3/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 128 + attention_resolutions: [ 4, 2, 1 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 4, 4 ] + num_head_channels: 32 + use_spatial_transformer: True + transformer_depth: 1 + context_dim: 4096 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 2 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 100 + validation: + target: ldm.data.volume_dataset.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_atten5.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_atten5.yaml new file mode 100644 index 0000000000000000000000000000000000000000..1a8939f57246c6f8f6972f119a430cddc7783a2e --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_atten5.yaml @@ -0,0 +1,93 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: hybrid + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-10-11T04-32-33_brain_slice_atten5/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 4, 2, 1 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 4, 4 ] + num_head_channels: 32 + use_spatial_transformer: True + transformer_depth: 1 + context_dim: 5 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 100 + validation: + target: ldm.data.volume_dataset.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_kidney.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_kidney.yaml new file mode 100644 index 0000000000000000000000000000000000000000..2328f6286a6c20ca6576fc3e8b3abd95b6e7c7aa --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_kidney.yaml @@ -0,0 +1,96 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-10-31T09-36-59_brain_slice_kidney/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset018_AbdomenAtlas2.0_base_kidney' + planner: 'nnUNetPlans_3d_fullres' + data_file: 'data_split/kidney/splits_final.json' + data_repeat: 4 + validation: + target: ldm.data.volume_dataset.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset018_AbdomenAtlas2.0_base_kidney' + planner: 'nnUNetPlans_3d_fullres' + data_file: 'data_split/kidney/splits_final.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_pancreas.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_pancreas.yaml new file mode 100644 index 0000000000000000000000000000000000000000..949617ab3329326a1f02091c320f574413742652 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_3d_pancreas.yaml @@ -0,0 +1,96 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: masked_volume + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: /storage/chenqi/code/GEM-3D-ct/logs/2024-10-31T09-43-34_brain_slice_pancreas/checkpoints/last.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 9 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.volume_dataset.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset017_AbdomenAtlas2.0_base_pancreas' + planner: 'nnUNetPlans_3d_fullres' + data_file: 'data_split/pancreas/splits_final.json' + data_repeat: 4 + validation: + target: ldm.data.volume_dataset.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset017_AbdomenAtlas2.0_base_pancreas' + planner: 'nnUNetPlans_3d_fullres' + data_file: 'data_split/pancreas/splits_final.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_base_stage1.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_base_stage1.yaml new file mode 100644 index 0000000000000000000000000000000000000000..37cfe5164f06d92ff0949e62be7c7d80f5f465a3 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_base_stage1.yaml @@ -0,0 +1,90 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 8 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: 'models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + train: + target: ldm.data.volume_dataset_base.volume_train + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset200_Abdomen' + data_repeat: 100 + validation: + target: ldm.data.volume_dataset_base.volume_val + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset200_Abdomen' + + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 20000 + max_images: 8 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_base_stage2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_base_stage2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e200c08d09a4d6664d0a910299cef4b52677c7c7 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_base_stage2.yaml @@ -0,0 +1,93 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: path-to-stage1-ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 8 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: 'models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + train: + target: ldm.data.volume_dataset_base.volume_train + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset200_Abdomen' + data_repeat: 100 + validation: + target: ldm.data.volume_dataset_base.volume_val + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset200_Abdomen' + + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 20000 + max_images: 8 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_slice.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_slice.yaml new file mode 100644 index 0000000000000000000000000000000000000000..4c3360c0cda853d0b0d3b53c1d6bb8b73e8dc8e5 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_slice.yaml @@ -0,0 +1,94 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: pos_id + cond_stage_trainable: false + conditioning_key: crossattn + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 4 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 4, 2, 1 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 4, 4 ] + num_head_channels: 32 + use_spatial_transformer: True + transformer_depth: 1 + context_dim: 128 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: 'models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: + target: ldm.modules.encoders.modules.PositionalEmbedder + params: + dim: 128 + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 5 + wrap: false + train: + target: ldm.data.slice_dataset.slice_train + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset200_Abdomen' + data_repeat: 300 + validation: + target: ldm.data.slice_dataset.slice_val + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset200_Abdomen' + + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 20000 + max_images: 8 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_stage1.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_stage1.yaml new file mode 100644 index 0000000000000000000000000000000000000000..645ae86af5143000dae304f9ac4a76d033a999b3 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_stage1.yaml @@ -0,0 +1,90 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: ref_and_volume_seg + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 12 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: 'models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + train: + target: ldm.data.volume_dataset.volume_train + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset200_Abdomen' + data_repeat: 100 + validation: + target: ldm.data.volume_dataset.volume_val + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset200_Abdomen' + + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 20000 + max_images: 8 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_stage2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_stage2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..6132ebfca21c0161b2fedad95f66608a3f3d0b51 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/abdomen_stage2.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: ref_and_volume_seg + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: path-to-stage1-ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 12 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: 'models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + train: + target: ldm.data.volume_dataset.volume_train + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset200_Abdomen' + data_repeat: 100 + validation: + target: ldm.data.volume_dataset.volume_val + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset200_Abdomen' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 20000 + max_images: 8 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_base_stage1.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_base_stage1.yaml new file mode 100644 index 0000000000000000000000000000000000000000..6b707fb63b865776dacd93eb2af8c8b67a92f75c --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_base_stage1.yaml @@ -0,0 +1,90 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 8 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: 'models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + train: + target: ldm.data.volume_dataset_base.volume_train + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset202_Brain' + data_repeat: 100 + validation: + target: ldm.data.volume_dataset_base.volume_val + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset202_Brain' + + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 20000 + max_images: 8 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_base_stage2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_base_stage2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..c22e3cae54b1c2b63ddd5e18f7129feb8c54bb3a --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_base_stage2.yaml @@ -0,0 +1,93 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: path-to-stage1-ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 8 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: 'models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + train: + target: ldm.data.volume_dataset_base.volume_train + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset202_Brain' + data_repeat: 100 + validation: + target: ldm.data.volume_dataset_base.volume_val + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset202_Brain' + + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 20000 + max_images: 8 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice.yaml new file mode 100644 index 0000000000000000000000000000000000000000..db27b89a55e936f1428e183e92bcbb971b77deb0 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice.yaml @@ -0,0 +1,90 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 16 + wrap: false + train: + target: ldm.data.slice_dataset.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 200 # 1000 100 + validation: + target: ldm.data.slice_dataset.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass.yaml new file mode 100644 index 0000000000000000000000000000000000000000..87b95745018d70ac2dc149d94f302b0ed54c29c2 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass.yaml @@ -0,0 +1,90 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset103_AbdomenAtlas2.0_base' + data_repeat: 4 # 1000 100 + validation: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset103_AbdomenAtlas2.0_base' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a87b5034aa78be063f904eff089a6edca0fbb610 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset103_AbdomenAtlas2.0_base' + data_repeat: 4 # 1000 100 + data_file: 'data_split/splits_final_new.json' + validation: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset103_AbdomenAtlas2.0_base' + data_repeat: 80 # 1000 100 + data_file: 'data_split/splits_final_new.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..cea75aeaed628f090b2b13bfb1b452f7d33dd196 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata2.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset103_AbdomenAtlas2.0_base' + data_repeat: 4 # 1000 100 + data_file: 'data_split/splits_final_all.json' + validation: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset103_AbdomenAtlas2.0_base' + data_repeat: 10 # 1000 100 + data_file: 'data_split/splits_final_all.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata3.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata3.yaml new file mode 100644 index 0000000000000000000000000000000000000000..ba7b7d038c08dee9c96533848b77ecba2facc29c --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata3.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset104_AbdomenAtlas2.0_1102' + data_repeat: 4 # 1000 100 + data_file: 'data_split/splits_final_all2.json' + validation: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset104_AbdomenAtlas2.0_1102' + data_repeat: 10 # 1000 100 + data_file: 'data_split/splits_final_all2.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata4.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata4.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e8236f13d1c8a0c3f02778fedf6944b13b99b51e --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata4.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset105_AbdomenAtlas2.0_1106' + data_repeat: 4 # 1000 100 + data_file: 'data_split/splits_final_all3.json' + validation: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset105_AbdomenAtlas2.0_1106' + data_repeat: 10 # 1000 100 + data_file: 'data_split/splits_final_all3.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata_kl4.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata_kl4.yaml new file mode 100644 index 0000000000000000000000000000000000000000..2f2a9a78fa498ad5bbbea7fb8d7ada95812564e1 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_13class_new_randomclass_alldata_kl4.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 128 + channels: 7 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 128 + in_channels: 7 # 4 + out_channels: 3 # 4 + model_channels: 192 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f4/model.ckpt' + embed_dim: 3 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 3 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 16 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass_kl4.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset104_AbdomenAtlas2.0_1102' + data_repeat: 4 # 1000 100 + data_file: 'data_split/splits_final_all2.json' + validation: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass_kl4.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset104_AbdomenAtlas2.0_1102' + data_repeat: 1 # 1000 100 + data_file: 'data_split/splits_final_all2.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100 + max_images: 24 + increase_log_steps: False + + trainer: + benchmark: True + # max_epochs: 400 diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class.yaml new file mode 100644 index 0000000000000000000000000000000000000000..f764ddc774430556a7492a417e5688f69a9485e0 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class.yaml @@ -0,0 +1,90 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 16 + wrap: false + train: + target: ldm.data.slice_dataset_abdomenatlas_resize.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset100_AbdomenAtlasX' + data_repeat: 4 # 1000 100 + validation: + target: ldm.data.slice_dataset_abdomenatlas_resize.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset100_AbdomenAtlasX' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class_new.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class_new.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0784ba552c5e3c3e9737cc69b736cff142329eac --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class_new.yaml @@ -0,0 +1,90 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.slice_dataset_abdomenatlas_28class.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset101_AbdomenAtlasInternal' + data_repeat: 4 # 1000 100 + validation: + target: ldm.data.slice_dataset_abdomenatlas_28class.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset101_AbdomenAtlasInternal' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class_new_randomclass.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class_new_randomclass.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b1386468645c74fb00cd6e2499f8b4128fe77c50 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class_new_randomclass.yaml @@ -0,0 +1,98 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 8 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 8 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/sd/qichen/full_ct_gen/latent-diffusion/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + # cond_stage_config: __is_first_stage__ + + cond_stage_config: + target: ldm.modules.encoders.modules.BERTEmbedder + params: + n_embed: 1280 + n_layer: 32 + +data: + target: main.DataModuleFromConfig + params: + batch_size: 2 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + labels: '/sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 # 100 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class_new_randomclass_2k.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class_new_randomclass_2k.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b724f5a02400388737a6d5975216c16b4b3576b4 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_28class_new_randomclass_2k.yaml @@ -0,0 +1,91 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset101_AbdomenAtlasInternal' + data_repeat: 4 # 1000 100 + validation: + target: ldm.data.slice_dataset_abdomenatlas_28class_randomclass.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset101_AbdomenAtlasInternal' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True + max_epochs: 200 diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten.yaml new file mode 100644 index 0000000000000000000000000000000000000000..1e7b20b53fa354c7bf821b0c697368fb1f02ed10 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten.yaml @@ -0,0 +1,90 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: crossattn # crossattn + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 4 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 16 + wrap: false + train: + target: ldm.data.slice_dataset.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 200 # 1000 100 + validation: + target: ldm.data.slice_dataset.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..3a84d911b03025a442926e16167d77602ddcc52b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten2.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: crossattn # crossattn + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 4 # 4 + out_channels: 4 # 4 + model_channels: 128 + attention_resolutions: [ 4, 2, 1 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 4, 4 ] + num_head_channels: 32 + use_spatial_transformer: True + transformer_depth: 1 + context_dim: 5 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 56 # 32 + num_workers: 16 + wrap: false + train: + target: ldm.data.slice_dataset.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 200 # 1000 100 + validation: + target: ldm.data.slice_dataset.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100 + max_images: 56 + increase_log_steps: False + + trainer: + benchmark: True + max_epochs: 200 \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten3.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten3.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b654f88a0b2b96874b7d32e9643d8fc8ae7c1a2b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten3.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: hybrid # crossattn + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 128 + attention_resolutions: [ 4, 2, 1 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 4, 4 ] + num_head_channels: 32 + use_spatial_transformer: True + transformer_depth: 1 + context_dim: 5 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 56 # 32 + num_workers: 16 + wrap: false + train: + target: ldm.data.slice_dataset.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 200 # 1000 100 + validation: + target: ldm.data.slice_dataset.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100 + max_images: 56 + increase_log_steps: False + + trainer: + benchmark: True + max_epochs: 200 diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten4.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten4.yaml new file mode 100644 index 0000000000000000000000000000000000000000..af302905d3c984295b6b777c4bb38213e23723f5 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten4.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: crossattn # crossattn + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 4 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 4, 2, 1 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 4, 4 ] + num_head_channels: 32 + use_spatial_transformer: True + transformer_depth: 1 + context_dim: 5 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 48 # 32 + num_workers: 16 + wrap: false + train: + target: ldm.data.slice_dataset.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 200 # 1000 100 + validation: + target: ldm.data.slice_dataset.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100 + max_images: 56 + increase_log_steps: False + + trainer: + benchmark: True + max_epochs: 200 diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten5.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten5.yaml new file mode 100644 index 0000000000000000000000000000000000000000..ae38ec78fd29f3f80d48e86b681e9fc1d0b7d1ff --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_atten5.yaml @@ -0,0 +1,92 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: hybrid # crossattn + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 4, 2, 1 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 4, 4 ] + num_head_channels: 32 + use_spatial_transformer: True + transformer_depth: 1 + context_dim: 5 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 48 # 32 + num_workers: 16 + wrap: false + train: + target: ldm.data.slice_dataset.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 200 # 1000 100 + validation: + target: ldm.data.slice_dataset.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100 + max_images: 56 + increase_log_steps: False + + trainer: + benchmark: True + max_epochs: 200 diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_ch384.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_ch384.yaml new file mode 100644 index 0000000000000000000000000000000000000000..01ed38b74795928fe5a438c21f15edb33c3a6689 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_ch384.yaml @@ -0,0 +1,90 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 384 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 16 + wrap: false + train: + target: ldm.data.slice_dataset.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 200 # 1000 100 + validation: + target: ldm.data.slice_dataset.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_ch512.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_ch512.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b8ac17442464075e48f803040701e1bb4a0a2dff --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_ch512.yaml @@ -0,0 +1,90 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 512 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 32 # 32 + num_workers: 16 + wrap: false + train: + target: ldm.data.slice_dataset.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 200 # 1000 100 + validation: + target: ldm.data.slice_dataset.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' + data_repeat: 80 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_kidney.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_kidney.yaml new file mode 100644 index 0000000000000000000000000000000000000000..6898365658ff7f724b73ba01bc274d44ff24150d --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_kidney.yaml @@ -0,0 +1,94 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 48 # 32 + num_workers: 16 + wrap: false + train: + target: ldm.data.slice_dataset.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset018_AbdomenAtlas2.0_base_kidney' + planner: 'nnUNetPlans_3d_fullres' + data_file: 'data_split/kidney/splits_final.json' + data_repeat: 20 # 1000 100 + validation: + target: ldm.data.slice_dataset.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset018_AbdomenAtlas2.0_base_kidney' + planner: 'nnUNetPlans_3d_fullres' + data_file: 'data_split/kidney/splits_final.json' + data_repeat: 20 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_pancreas.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_pancreas.yaml new file mode 100644 index 0000000000000000000000000000000000000000..8673c4b8f22f107824ed008044bb59a3e2f31f82 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_slice_pancreas.yaml @@ -0,0 +1,94 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: slice_data + cond_stage_key: masked_slice + cond_stage_trainable: false + conditioning_key: concat # crossattn + image_size: 64 + channels: 9 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 9 # 4 + out_channels: 4 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: '/storage/chenqi/code/latent-diffusion-main/models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 48 # 32 + num_workers: 16 + wrap: false + train: + target: ldm.data.slice_dataset.slice_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset017_AbdomenAtlas2.0_base_pancreas' + planner: 'nnUNetPlans_3d_fullres' + data_file: 'data_split/pancreas/splits_final.json' + data_repeat: 50 # 1000 100 + validation: + target: ldm.data.slice_dataset.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset017_AbdomenAtlas2.0_base_pancreas' + planner: 'nnUNetPlans_3d_fullres' + data_file: 'data_split/pancreas/splits_final.json' + data_repeat: 20 # 1000 100 + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 500 + max_images: 32 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_stage1.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_stage1.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a3ab2756587418343eef51b37e1d70f01577a216 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_stage1.yaml @@ -0,0 +1,90 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: ref_and_volume_seg + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 12 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: 'models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + train: + target: ldm.data.volume_dataset.volume_train + params: + data_root: '/storage/chenqi/data/gem_3d_data/nnUNet_preprocessed/' + data_name: 'Dataset202_Brain' + data_repeat: 100 + validation: + target: ldm.data.volume_dataset.volume_val + params: + data_root: '/storage/chenqi/data/gem_3d_data/nnUNet_preprocessed/' + data_name: 'Dataset202_Brain' + + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 20000 + max_images: 8 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_stage2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_stage2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..c90408472e62f1a13e009e02e99ba7f07dac72e6 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/brain_stage2.yaml @@ -0,0 +1,93 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: ref_and_volume_seg + cond_stage_trainable: false + conditioning_key: concat + image_size: 64 + channels: 4 + monitor: val/loss_simple_ema + scale_factor: 0.18215 + use_ema: False + ckpt_path: path-to-stage1-ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 12 + out_channels: 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: false + transformer_depth: 1 + use_checkpoint: True + legacy: False + + first_stage_config: + target: ldm.models.autoencoder.AutoencoderKL + params: + ckpt_path: 'models/first_stage_models/kl-f8/model.ckpt' + embed_dim: 4 + monitor: val/rec_loss + ddconfig: + double_z: true + z_channels: 4 + resolution: 256 + in_channels: 3 + out_ch: 3 + ch: 128 + ch_mult: + - 1 + - 2 + - 4 + - 4 + num_res_blocks: 2 + attn_resolutions: [] + dropout: 0.0 + lossconfig: + target: torch.nn.Identity + cond_stage_config: __is_first_stage__ + +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + train: + target: ldm.data.volume_dataset.volume_train + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset202_Brain' + data_repeat: 100 + validation: + target: ldm.data.volume_dataset.volume_val + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset202_Brain' + + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 20000 + max_images: 8 + increase_log_steps: False + + trainer: + benchmark: True \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask.yaml new file mode 100644 index 0000000000000000000000000000000000000000..34cb847d9aaf47d2189e1d446bdc81ecad1024bf --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask.yaml @@ -0,0 +1,67 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.0 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 8 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + labels: '/sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 # 100 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b4038ffac8caf74db49fdb7dc87ad0dbd03c4745 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask2.yaml @@ -0,0 +1,67 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 8 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + labels: '/sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 # 100 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask3.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask3.yaml new file mode 100644 index 0000000000000000000000000000000000000000..f57bb7c6736731ab039bb1264b1f4549f2da43b7 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask3.yaml @@ -0,0 +1,67 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.51 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 8 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + labels: '/sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 # 100 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask4.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask4.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b4038ffac8caf74db49fdb7dc87ad0dbd03c4745 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_2d_with_body_mask4.yaml @@ -0,0 +1,67 @@ +model: + base_learning_rate: 5.0e-7 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 8 # 32 + num_workers: 12 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + labels: '/sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 # 100 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask.yaml new file mode 100644 index 0000000000000000000000000000000000000000..1295d796159f0d5e82f66a5d48c77a4605f6bab5 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask.yaml @@ -0,0 +1,70 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: False + ckpt_path: /sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae-all3/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000317.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + labels: '/sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..15cbf17e7be4ad386b574e2e9dab39d49219f8a8 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask2.yaml @@ -0,0 +1,70 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: False + ckpt_path: /sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae-all3-fix/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000317.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 8 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + labels: '/sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask2_finetune.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask2_finetune.yaml new file mode 100644 index 0000000000000000000000000000000000000000..033260c6cfa4181cbd87dc28c252aa8b0e2090ef --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask2_finetune.yaml @@ -0,0 +1,71 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: False + ckpt_path: /sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae/logs/full_ct_2d_with_body_mask4/checkpoints/epoch=000833.ckpt + # ckpt_path: /sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae/logs/full_ct_2d_with_body_mask/checkpoints/base.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + labels: '/sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask2_finetune2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask2_finetune2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..621c875b2faff63b17e93a097d1f41b7b0b249e9 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask2_finetune2.yaml @@ -0,0 +1,70 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: False + ckpt_path: /sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae/logs/full_ct_2d_with_body_mask4/checkpoints/epoch=000833.ckpt + load_only_unet: True + fix_t: False + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + labels: '/sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask_finetune.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask_finetune.yaml new file mode 100644 index 0000000000000000000000000000000000000000..14b225d0b231cd329c57257b5b441338773e7631 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask_finetune.yaml @@ -0,0 +1,70 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.0 + use_ema: False + ckpt_path: /sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae/logs/full_ct_2d_with_body_mask/checkpoints/base.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + labels: '/sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask_finetune2.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask_finetune2.yaml new file mode 100644 index 0000000000000000000000000000000000000000..37017b47fe7ee6ef9c2e9f0efe53fe98bf8adb09 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/full_ct_3d_with_body_mask_finetune2.yaml @@ -0,0 +1,70 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.0 + use_ema: False + ckpt_path: /sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae/logs/full_ct_2d_with_body_mask/checkpoints/base.ckpt + load_only_unet: True + fix_t: False + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + labels: '/sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/infer_train_multi.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/infer_train_multi.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0ce5ddd09583a4d2131407dd4364d9bb746b8d17 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/infer_train_multi.yaml @@ -0,0 +1,70 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: False + ckpt_path: /sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae-all/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000081.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference_eval.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + json_file: './inference_train.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/infer_valid.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/infer_valid.yaml new file mode 100644 index 0000000000000000000000000000000000000000..9465d175240738f4eff921a26c344b9ad4b2d00c --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/infer_valid.yaml @@ -0,0 +1,70 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: False + ckpt_path: /sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae-all/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000081.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference_eval.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + json_file: '/sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae-all3-fix/inference_valid.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/infer_valid_multi.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/infer_valid_multi.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a32f31199b24341cbab0596fe5b2db08970b0c80 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/infer_valid_multi.yaml @@ -0,0 +1,70 @@ +model: + base_learning_rate: 5.0e-6 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.0120 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + # cond_stage_trainable: false + conditioning_key: crossattn # crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: False + ckpt_path: /sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae-all/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000081.ckpt + load_only_unet: True + fix_t: True + + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 32 # 4 + out_channels: 16 # 4 + model_channels: 224 + attention_resolutions: [ 8, 4, 2 ] + num_res_blocks: 2 + channel_mult: [ 1, 2, 3, 4 ] + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: True + legacy: False + +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 16 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/train_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv' + + validation: + target: ldm.data.ct_clip_data_inference_eval.CTReportDatasetinfer + params: + data_folder: '/sd/shuhan/CT-RATE/dataset/valid_fixed' + csv_file: '/sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv' + json_file: './inference_valid.json' + +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: False + + trainer: + benchmark: True diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen.yaml new file mode 100644 index 0000000000000000000000000000000000000000..3919299787898eaa15bf2594bc571d4ac998de98 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen.yaml @@ -0,0 +1,11 @@ +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + validation: + target: ldm.data.slice_dataset_infer.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen3d.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen3d.yaml new file mode 100644 index 0000000000000000000000000000000000000000..f87d78f163e381bcd8ddcd03ad061a5e8c19c5c7 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen3d.yaml @@ -0,0 +1,12 @@ +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + validation: + # target: ldm.data.volume_dataset_infer.volume_val + target: ldm.data.volume_dataset_infer.volume_train + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen_28class_randomclass.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen_28class_randomclass.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b1fda1e8074104293585b7fb56c408b640997a90 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen_28class_randomclass.yaml @@ -0,0 +1,11 @@ +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + validation: + target: ldm.data.slice_dataset_infer_28class_randomclass.slice_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen_test.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen_test.yaml new file mode 100644 index 0000000000000000000000000000000000000000..99590484c0d32f7260e344f2245da263aa6284a3 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_abdomen_test.yaml @@ -0,0 +1,11 @@ +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + validation: + target: ldm.data.volume_dataset.volume_test + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset201_Abdomen_test' \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_brain.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_brain.yaml new file mode 100644 index 0000000000000000000000000000000000000000..1e52b9de4d46b2d3d50575996a06db5f1a80ef6f --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_brain.yaml @@ -0,0 +1,11 @@ +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + validation: + target: ldm.data.volume_dataset.volume_test + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset202_Brain' \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_brain_test.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_brain_test.yaml new file mode 100644 index 0000000000000000000000000000000000000000..80bcabf1bd9acfe663f6b4bb835ea575891d37ab --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/inference_brain_test.yaml @@ -0,0 +1,11 @@ +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + validation: + target: ldm.data.volume_dataset.volume_test + params: + data_root: './data/nnUNet_preprocessed/' + data_name: 'Dataset203_Brain_test' \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/mask_generation.yaml b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/mask_generation.yaml new file mode 100644 index 0000000000000000000000000000000000000000..09ed691b3a1209780a1d5644e9b1e3e00ad8908e --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/configs/latent-diffusion/mask_generation.yaml @@ -0,0 +1,11 @@ +data: + target: main.DataModuleFromConfig + params: + batch_size: 1 + num_workers: 5 + wrap: false + validation: + target: ldm.data.mask_generation.volume_val + params: + data_root: '/storage/chenqi/data/our_data/nnUNet_preprocessed/' + data_name: 'Dataset010_MSD_Liver' \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/data_split.py b/Abnormal-CT-Generation-MultiDisease/data_split.py new file mode 100644 index 0000000000000000000000000000000000000000..93e6e36f375bf381c00ef6dd8eadcd2138f27c7f --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/data_split.py @@ -0,0 +1,43 @@ +import random +import os + +# 设定文件路径 +train_file = 'train.txt' # 训练集输出文件 +eval_file = 'eval.txt' # 验证集输出文件 +test_file = 'test.txt' # 测试集输出文件 + +# 读取数据 +data = sorted(os.listdir('/storage/chenqi/data/BraTS_2019_Data_Training/All')) + +# 随机打乱数据 +random.shuffle(data) + +# 计算各个数据集的大小 +train_size = 290 +eval_size = 8 +test_size = 37 + +# 划分数据集 +train_data = data[:train_size] +eval_data = data[train_size:train_size + eval_size] +test_data = data[train_size + eval_size:] + +# 保存到txt文件 +with open(train_file, 'w') as file: + for i in train_data: + file.write(i) + file.write('\n') + +with open(eval_file, 'w') as file: + for i in eval_data: + file.write(i) + file.write('\n') + + +with open(test_file, 'w') as file: + for i in test_data: + file.write(i) + file.write('\n') + + +print(f"数据集已划分完成,并分别保存为: {train_file}, {eval_file}, {test_file}") diff --git a/Abnormal-CT-Generation-MultiDisease/environment.yml b/Abnormal-CT-Generation-MultiDisease/environment.yml new file mode 100644 index 0000000000000000000000000000000000000000..9fd50656e9218f87ed5c58ad72995d1c9b9dac7e --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/environment.yml @@ -0,0 +1,40 @@ +name: gem_ct +channels: + - pytorch + - defaults +dependencies: + - cudatoolkit=11.3 + - pip=22.3.1 + - python=3.9.18 + - pytorch=2.0.0 + - torchaudio=2.0.0 + - torchtriton=2.0.0 + - torchvision=0.15.0 + - yaml=0.2.5 + - pip: + - argparse==1.4.0 + - easydict==1.11 + - einops==0.7.0 + - imageio==2.31.6 + - lightning-utilities==0.9.0 + - lpips==0.1.4 + - nibabel==5.2.0 + - numpy==1.26.1 + - omegaconf==2.1.1 + - open-clip-torch==2.7.0 + - opencv-python==4.8.1.78 + - pandas==2.1.2 + - pyparsing==3.1.1 + - pytorch-lightning==1.4.2 + - scikit-image==0.22.0 + - scikit-learn==1.3.2 + - scipy==1.11.3 + - seaborn==0.13.0 + - test-tube + - tensorboard==2.15.1 + - tensorboardx==2.6.2.2 + - torch==2.1.0 + - torchmetrics==0.5.0 + - tqdm==4.66.1 + - transformers==4.29.2 + - volumentations-3D diff --git a/Abnormal-CT-Generation-MultiDisease/inference_abdomen_our_3d.py b/Abnormal-CT-Generation-MultiDisease/inference_abdomen_our_3d.py new file mode 100644 index 0000000000000000000000000000000000000000..6dc89e79a348351cdf30bee6b81abc32220762f2 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/inference_abdomen_our_3d.py @@ -0,0 +1,174 @@ +import glob +import nibabel as nib +import numpy as np +import os +import torch + +from einops import rearrange +from omegaconf import OmegaConf +from torch.utils.data import DataLoader +from tqdm import tqdm + +from ldm.util import instantiate_from_config +import argparse + +def compute_orientation(init_axcodes, final_axcodes): + """ + A thin wrapper around ``nib.orientations.ornt_transform`` + + :param init_axcodes: Initial orientation codes + :param final_axcodes: Target orientation codes + :return: orientations array, start_ornt, end_ornt + """ + ornt_init = nib.orientations.axcodes2ornt(init_axcodes) + ornt_fin = nib.orientations.axcodes2ornt(final_axcodes) + + ornt_transf = nib.orientations.ornt_transform(ornt_init, ornt_fin) + + return ornt_transf, ornt_init, ornt_fin + +def do_reorientation(data_array, init_axcodes, final_axcodes): + """ + source: https://niftynet.readthedocs.io/en/dev/_modules/niftynet/io/misc_io.html#do_reorientation + Performs the reorientation (changing order of axes) + + :param data_array: 3D Array to reorient + :param init_axcodes: Initial orientation + :param final_axcodes: Target orientation + :return data_reoriented: New data array in its reoriented form + """ + ornt_transf, ornt_init, ornt_fin = compute_orientation(init_axcodes, final_axcodes) + if np.array_equal(ornt_init, ornt_fin): + return data_array + + return nib.orientations.apply_orientation(data_array, ornt_transf) + +parser = argparse.ArgumentParser() +parser.add_argument( + "-t", + "--time_steps", + type=int, + default=20, +) +args = parser.parse_args() +ddim_steps=args.time_steps +# breakpoint() + +logdir = 'logs/full_ct_2d_with_body_mask' +ckpt = os.path.join(logdir, "checkpoints", "epoch=000068.ckpt") + +configs_file = "configs/latent-diffusion/full_ct_2d_with_body_mask.yaml" +configs = OmegaConf.load(configs_file) +model = instantiate_from_config(configs.model) +model.init_from_ckpt(ckpt) +model.eval() + +device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") +print("Using device", device) +model = model.to(device) + +config = OmegaConf.load('./configs/latent-diffusion/full_ct_2d_with_body_mask.yaml') +data = instantiate_from_config(config.data) +data.prepare_data() +data.setup() + +save_path = f'3d_results_step{ddim_steps}_train_latest' +save_path = os.path.join(logdir, save_path) +if not os.path.exists(save_path): + os.makedirs(save_path) + +val_dataset = data.datasets['validation'] +batch_size = 1 +valloader = DataLoader(val_dataset, batch_size=batch_size, shuffle=False, num_workers=2, pin_memory=True) +val_num = len(val_dataset) +save_gt = True + +for idx, data in tqdm(enumerate(valloader)): + + if idx >= val_num: + break + + name=data['name'][0].split('.')[0] + volume_data = data['volume_data'] + volume_seg = data['volume_seg'] + # breakpoint() + + window_length = 65 + latent_lenght=17 + h = 1 + slice_num =volume_data.shape[2] + result = torch.zeros((batch_size, latent_lenght, 16, 64, 64)).cuda() + + + upper_iters = (slice_num-h) // (window_length-h)+1 if (slice_num-h)%(window_length-h) != 0 else (slice_num-h) // (window_length-h) + print('upper_iters', upper_iters) + # breakpoint() + for i in range(upper_iters): + print('i', i) + input_data={} + if i == upper_iters-1: + input_data['name'] = data['name'] + input_data['volume_data'] = data['volume_data'][:, :, -window_length:].to(device) + input_data['volume_seg'] = data['volume_seg'][:, :, -window_length:].to(device) + input_data['input_text'] = data['input_text'] + else: + input_data['volume_data'] = data['volume_data'][:, :, i*window_length-i*h:(i+1)*window_length-i*h].to(device) + input_data['volume_seg'] = data['volume_seg'][:, :, i*window_length-i*h:(i+1)*window_length-i*h].to(device) + input_data['input_text'] = data['input_text'] + # breakpoint() + with torch.no_grad(): + _, c = model.get_input(input_data, model.first_stage_key) + # breakpoint() + if i == 0: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps) + else: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps, previous=x_minus1) + + samples_i = rearrange(samples_i, '(b z) c h w -> b z c h w', z=latent_lenght) + + if i == upper_iters-1: + result[:, -latent_lenght+h:] = samples_i[:,h:,...] + else: + if i == 0: + result[:, :latent_lenght] = samples_i + else: + result[:, i*latent_lenght-i*h+h:(i+1)*latent_lenght-i*h] = samples_i[:, h:] + x_minus1 = samples_i[:, -h:,...] + # breakpoint() + # result = rearrange(result, 'b z c h w -> (b z) c h w') + result = result.permute(0,2,1,3,4) + x_result = torch.zeros((3,slice_num,512,512)) + dec_unit = 65 + num_dec_iter = slice_num // dec_unit + 1 if slice_num % dec_unit != 0 else slice_num // dec_unit + for i in range(num_dec_iter): + if i == num_dec_iter - 1: + x_result[:,-dec_unit:] = model.decode_first_stage(result[:,-dec_unit:])[0] + x_result[:,i*dec_unit:(i+1)*dec_unit] = model.decode_first_stage(result[:,i*dec_unit:(i+1)*dec_unit])[0] + x_result = x_result*2 + x_result[x_result>1.0] = 1.0 + x_result[x_result<-1.0] = -1.0 + x_result = (x_result+1)/2 + # breakpoint() + + x_result_ = x_result.mean(axis=0).detach().cpu().numpy() + # x_result = x_result[0,:,0,...].detach().cpu().numpy() + # breakpoint() + x_result = x_result_.transpose(1,2,0) + # x_result = np.rot90(x_result, k=1, axes=(0,1)) + # x_result = np.flip(x_result,axis=(0,1)) + # import imageio as io + # io.imsave('exp.png', (x_result[:,:,400]*255).astype(np.uint8)) + + # breakpoint() + ref_root = '/sd/shuhan/CT-RATE/dataset/valid_fixed' + ref_nii = os.path.join(ref_root, name.split('_')[0]+'_'+name.split('_')[1], name.split('_')[0]+'_'+name.split('_')[1]+'_'+name.split('_')[2],name+'.nii.gz') + affine = nib.load(ref_nii).affine + + x_result = x_result*2000.0 - 1000.0 + data_path = os.path.join(save_path, str(f'{name}.nii.gz')) + data_nii = nib.Nifti1Image(x_result.astype(np.int16), affine) + + nib.save(data_nii, data_path) + + breakpoint() + diff --git a/Abnormal-CT-Generation-MultiDisease/inference_abdomen_our_3d_new4.py b/Abnormal-CT-Generation-MultiDisease/inference_abdomen_our_3d_new4.py new file mode 100644 index 0000000000000000000000000000000000000000..c27315b0d827d32e29b730e3d1d2c8cd46492fab --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/inference_abdomen_our_3d_new4.py @@ -0,0 +1,324 @@ +import glob +import nibabel as nib +import numpy as np +import os +import torch + +from einops import rearrange +from omegaconf import OmegaConf +from torch.utils.data import DataLoader +from tqdm import tqdm + +from ldm.util import instantiate_from_config +import argparse + +def compute_orientation(init_axcodes, final_axcodes): + """ + A thin wrapper around ``nib.orientations.ornt_transform`` + + :param init_axcodes: Initial orientation codes + :param final_axcodes: Target orientation codes + :return: orientations array, start_ornt, end_ornt + """ + ornt_init = nib.orientations.axcodes2ornt(init_axcodes) + ornt_fin = nib.orientations.axcodes2ornt(final_axcodes) + + ornt_transf = nib.orientations.ornt_transform(ornt_init, ornt_fin) + + return ornt_transf, ornt_init, ornt_fin + +def do_reorientation(data_array, init_axcodes, final_axcodes): + """ + source: https://niftynet.readthedocs.io/en/dev/_modules/niftynet/io/misc_io.html#do_reorientation + Performs the reorientation (changing order of axes) + + :param data_array: 3D Array to reorient + :param init_axcodes: Initial orientation + :param final_axcodes: Target orientation + :return data_reoriented: New data array in its reoriented form + """ + ornt_transf, ornt_init, ornt_fin = compute_orientation(init_axcodes, final_axcodes) + if np.array_equal(ornt_init, ornt_fin): + return data_array + + return nib.orientations.apply_orientation(data_array, ornt_transf) + +def compute_fusion_boundaries(upper_iters, latent_length, decode_ratio=4, blend_width=16): + """ + Compute fusion boundaries based on result segments + + :param upper_iters: Number of iterations in the generation loop + :param latent_length: Length of latent dimension (17) + :param decode_ratio: Ratio between decoded and latent dimensions (4) + :param blend_width: Width of blending region on each side (16 slices) + :return: List of boundary dictionaries for fusion + """ + boundaries = [] + + for i in range(1, upper_iters): + # Calculate boundary position in latent space + latent_boundary = i * latent_length + + # Convert to decoded space (multiply by decode_ratio) + decoded_boundary = latent_boundary * decode_ratio + 1 + + # Define blending region: boundary ± blend_width + blend_start = max(0, decoded_boundary - blend_width) + blend_end = decoded_boundary + blend_width + + boundaries.append({ + 'boundary': decoded_boundary, + 'blend_start': blend_start, + 'blend_end': blend_end + }) + + return boundaries + + +def alternating_volume_fusion_with_boundaries(vol1, vol2, boundaries, offset=32, replace_width=16): + """ + Fuse two volumes by replacing regions around boundaries + + :param vol1: First volume with shape (x, y, z) - used as base + :param vol2: Second volume with shape (x, y, z-offset), offset by 32 slices + :param boundaries: List of boundary dictionaries from compute_fusion_boundaries + :param offset: Offset between vol1 and vol2 (default 32) + :param replace_width: Width on each side of boundary to replace (default 16) + :return: Fused volume with vol2 replacing vol1 at boundary regions + """ + x, y, total_slices = vol1.shape + fused = vol1.copy() # Start with vol1 as base + + # Sort boundaries by position + sorted_boundaries = sorted(boundaries, key=lambda b: b['boundary']) + + print(f"Fusion starting with vol1 as base") + + for i, boundary_info in enumerate(sorted_boundaries): + boundary = boundary_info['boundary'] + + # Define replacement region: boundary ± replace_width + replace_start = max(0, boundary - replace_width) + replace_end = min(boundary + replace_width, total_slices) + + # Map to vol2 coordinates + vol2_start = replace_start - offset + vol2_end = replace_end - offset + + # Replace this region with vol2 if valid + if vol2_start >= 0 and vol2_end <= vol2.shape[2]: + fused[:, :, replace_start:replace_end] = vol2[:, :, vol2_start:vol2_end] + print(f" Boundary {i+1}: z={boundary}, replaced z[{replace_start}:{replace_end}] with vol2[{vol2_start}:{vol2_end}]") + else: + print(f" Boundary {i+1}: z={boundary}, skipped (vol2 out of range)") + + print(f"Fusion complete with {len(sorted_boundaries)} boundaries") + + return fused + +parser = argparse.ArgumentParser() +parser.add_argument( + "-t", + "--time_steps", + type=int, + default=20, +) +args = parser.parse_args() +ddim_steps=args.time_steps + +logdir = 'logs/full_ct_3d_with_body_mask2' +ckpt = os.path.join(logdir, "checkpoints", "epoch=000061.ckpt") + +configs_file = "configs/latent-diffusion/full_ct_3d_with_body_mask2.yaml" +configs = OmegaConf.load(configs_file) +model = instantiate_from_config(configs.model) +model.init_from_ckpt(ckpt) +model.eval() + +device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") +print("Using device", device) +model = model.to(device) + +config = OmegaConf.load('./configs/latent-diffusion/full_ct_3d_with_body_mask2.yaml') +data = instantiate_from_config(config.data) +data.prepare_data() +data.setup() + +save_path = f'3d_results_step{ddim_steps}_train_latest' +save_path = os.path.join(logdir, save_path) +if not os.path.exists(save_path): + os.makedirs(save_path) + +val_dataset = data.datasets['validation'] +batch_size = 1 +valloader = DataLoader(val_dataset, batch_size=batch_size, shuffle=False, num_workers=2, pin_memory=True) +val_num = len(val_dataset) +save_gt = True + +for idx, data in tqdm(enumerate(valloader)): + + if idx >= val_num: + break + + name=data['name'][0].split('.')[0] + volume_data = data['volume_data'] + volume_seg = data['volume_seg'] + + window_length = 65 + latent_lenght=17 + slice_num = volume_data.shape[2] + + upper_iters = slice_num // window_length + 1 if slice_num % window_length != 0 else slice_num // window_length + result = torch.zeros((batch_size, upper_iters*latent_lenght, 16, 64, 64)).cuda() + print('upper_iters', upper_iters) + + for i in range(upper_iters): + print('i', i) + input_data={} + if i == upper_iters-1: + input_data['name'] = data['name'] + input_data['volume_data'] = data['volume_data'][:, :, -window_length:].to(device) + input_data['volume_seg'] = data['volume_seg'][:, :, -window_length:].to(device) + input_data['input_text'] = data['input_text'] + else: + input_data['volume_data'] = data['volume_data'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['volume_seg'] = data['volume_seg'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['input_text'] = data['input_text'] + + with torch.no_grad(): + _, c = model.get_input(input_data, model.first_stage_key) + + if i == 0: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps) + else: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps, previous=x_minus1) + + samples_i = rearrange(samples_i, '(b z) c h w -> b z c h w', z=latent_lenght) + + if i == upper_iters-1: + result[:, -latent_lenght:] = samples_i + else: + result[:, i*latent_lenght:(i+1)*latent_lenght] = samples_i + x_minus1 = samples_i[:, -1:, ...] # Take last slice for next iteration + + result = result.permute(0,2,1,3,4) + x_result = torch.zeros((3,slice_num,512,512)) + dec_unit = 65 + dec_latent_unit=17 + num_dec_iter = slice_num // dec_unit + 1 if slice_num % dec_unit != 0 else slice_num // dec_unit + + for i in range(num_dec_iter): + if i == num_dec_iter - 1: + remaining_slices = slice_num % dec_unit if slice_num % dec_unit != 0 else dec_unit + x_result[:, -remaining_slices:] = model.decode_first_stage(result[:, :, -latent_lenght:])[0][:, -remaining_slices:] + else: + x_result[:, i*dec_unit:(i+1)*dec_unit] = model.decode_first_stage(result[:, :, i*latent_lenght:(i+1)*latent_lenght])[0] + + x_result = x_result*2 + x_result[x_result>1.0] = 1.0 + x_result[x_result<-1.0] = -1.0 + x_result = (x_result+1)/2 + x_result_ = x_result.mean(axis=0).detach().cpu().numpy() + x_result = x_result_.transpose(1,2,0) + + # Process second volume with offset + data2={} + data2['volume_data'] = data['volume_data'][:, :, 32:] + data2['volume_seg'] = data['volume_seg'][:, :, 32:] + data2['name'] = data['name'] + data2['input_text'] = data['input_text'] + + volume_data = data2['volume_data'] + volume_seg = data2['volume_seg'] + + window_length = 65 + latent_lenght=17 + slice_num2 = volume_data.shape[2] + + upper_iters2 = slice_num2 // window_length + 1 if slice_num2 % window_length != 0 else slice_num2 // window_length + result2 = torch.zeros((batch_size, upper_iters2*latent_lenght, 16, 64, 64)).cuda() + print('upper_iters2', upper_iters2) + + for i in range(upper_iters2): + print('i', i) + input_data={} + if i == upper_iters2-1: + input_data['name'] = data2['name'] + input_data['volume_data'] = data2['volume_data'][:, :, -window_length:].to(device) + input_data['volume_seg'] = data2['volume_seg'][:, :, -window_length:].to(device) + input_data['input_text'] = data2['input_text'] + else: + input_data['volume_data'] = data2['volume_data'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['volume_seg'] = data2['volume_seg'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['input_text'] = data2['input_text'] + + with torch.no_grad(): + _, c = model.get_input(input_data, model.first_stage_key) + + if i == 0: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps) + else: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps, previous=x_minus1) + + samples_i = rearrange(samples_i, '(b z) c h w -> b z c h w', z=latent_lenght) + + if i == upper_iters2-1: + result2[:, -latent_lenght:] = samples_i + else: + result2[:, i*latent_lenght:(i+1)*latent_lenght] = samples_i + x_minus1 = samples_i[:, -1:, ...] # Take last slice for next iteration + + result2 = result2.permute(0,2,1,3,4) + x_result2 = torch.zeros((3,slice_num2,512,512)) + dec_unit = 65 + dec_latent_unit=17 + num_dec_iter = slice_num2 // dec_unit + 1 if slice_num2 % dec_unit != 0 else slice_num2 // dec_unit + + for i in range(num_dec_iter): + if i == num_dec_iter - 1: + remaining_slices = slice_num2 % dec_unit if slice_num2 % dec_unit != 0 else dec_unit + x_result2[:, -remaining_slices:] = model.decode_first_stage(result2[:, :, -latent_lenght:])[0][:, -remaining_slices:] + else: + x_result2[:, i*dec_unit:(i+1)*dec_unit] = model.decode_first_stage(result2[:, :, i*latent_lenght:(i+1)*latent_lenght])[0] + + x_result2 = x_result2*2 + x_result2[x_result2>1.0] = 1.0 + x_result2[x_result2<-1.0] = -1.0 + x_result2 = (x_result2+1)/2 + x_result2_ = x_result2.mean(axis=0).detach().cpu().numpy() + x_result2 = x_result2_.transpose(1,2,0) + + # Compute fusion boundaries based on result segments + boundaries = compute_fusion_boundaries( + upper_iters=upper_iters, + latent_length=16, + decode_ratio=4, + blend_width=16 + ) + + # Fuse volumes: start with vol1, replace regions around boundaries with vol2 + print("Fusing volumes: replacing boundary regions with vol2...") + x_result_fused = alternating_volume_fusion_with_boundaries( + x_result, + x_result2, + boundaries, + offset=32, + replace_width=16 + ) + + # Load reference and save fused result + ref_root = '/sd/shuhan/CT-RATE/dataset/valid_fixed' + ref_nii = os.path.join(ref_root, name.split('_')[0]+'_'+name.split('_')[1], name.split('_')[0]+'_'+name.split('_')[1]+'_'+name.split('_')[2],name+'.nii.gz') + if not os.path.exists(ref_nii): + ref_root = '/sd/shuhan/CT-RATE/dataset/train_fixed' + ref_nii = os.path.join(ref_root, name.split('_')[0]+'_'+name.split('_')[1], name.split('_')[0]+'_'+name.split('_')[1]+'_'+name.split('_')[2],name+'.nii.gz') + + affine = nib.load(ref_nii).affine + + x_result_fused = x_result_fused*2000.0 - 1000.0 + data_path = os.path.join(save_path, str(f'{name}_fused.nii.gz')) + data_nii = nib.Nifti1Image(x_result_fused.astype(np.int16), affine) + + nib.save(data_nii, data_path) + print(f"Saved fused volume: {data_path}") + # breakpoint() \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/inference_full_ct_2d_with_body_mask.py b/Abnormal-CT-Generation-MultiDisease/inference_full_ct_2d_with_body_mask.py new file mode 100644 index 0000000000000000000000000000000000000000..4569e01d52605ad58bd68788957f6564158db1b3 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/inference_full_ct_2d_with_body_mask.py @@ -0,0 +1,80 @@ +import glob +import nibabel as nib +import numpy as np +import os +import torch +from skimage import io +from einops import rearrange +from omegaconf import OmegaConf +from torch.utils.data import DataLoader +from tqdm import tqdm + +from ldm.util import instantiate_from_config + + +logdir = 'logs/full_ct_2d_with_body_mask2/' +ckpt = os.path.join(logdir, "checkpoints", "epoch=000317.ckpt") + +configs_file = sorted(glob.glob(os.path.join(logdir, "configs/*.yaml")))[1] +configs = OmegaConf.load(configs_file) +model = instantiate_from_config(configs.model) +model.init_from_ckpt(ckpt) +model.eval() + +device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") +print("Using device", device) +model = model.to(device) + +config = OmegaConf.load('./configs/latent-diffusion/full_ct_2d_with_body_mask2.yaml') +data = instantiate_from_config(config.data) +data.prepare_data() +data.setup() + +save_path = 'inference' +save_path = os.path.join(logdir, save_path) +if not os.path.exists(save_path): + os.makedirs(save_path) + +val_dataset = data.datasets['validation'] +batch_size = 1 +valloader = DataLoader(val_dataset, batch_size=batch_size, shuffle=False, num_workers=0, pin_memory=True) +val_num = len(val_dataset) +save_gt = True +# val_num = 10 +# breakpoint() +for idx, data in tqdm(enumerate(valloader)): + # if idx >= val_num: + # break + name=data['name'][0].split('.')[0] + data['volume_data'] = data['volume_data'].to(device) + data['volume_seg'] = data['volume_seg'].to(device) + input_text = data['input_text'] + # .to(device) + + x, c = model.get_input(data, model.first_stage_key) + # breakpoint() + samples, _ = model.sample_log(cond=c, batch_size=17, ddim=True, eta=1., ddim_steps=100) + samples=samples.permute(1,0,2,3) + + res = model.decode_first_stage(samples[None,:]) + res = res*2 + # breakpoint() + res[res>1.0] = 1.0 + res[res<-1.0] = -1.0 + res = (res+1)/2 + # breakpoint() + res = res[0].mean(axis=0).detach().cpu().numpy() + # slice_ct=(slice_ct+1)/2 + # slice_seg=(slice_seg+1)/2 + for z_slice in range(res.shape[0]): + # input_img = np.clip(np.repeat(slice_ct[0,0].detach().cpu().numpy(), 3, 0),0,1) * 255 + res_slice = np.repeat(res[z_slice][None,], 3, 0).transpose(1,2,0)* 255 + # slice_seg = np.clip(np.repeat(slice_seg[0,0].detach().cpu().numpy(), 3, 0),0,1) * 255 + + io.imsave(os.path.join(save_path, str(name) + f'_sample_{z_slice}.png'), res_slice.astype(np.uint8)) + # io.imsave(os.path.join(save_path, str(name) + f'_input_img_{z_slice}.png'), input_img.astype(np.uint8)) + # io.imsave(os.path.join(save_path, str(name) + f'_seg_{z_slice}.png'), slice_seg.astype(np.uint8)) + + break + + diff --git a/Abnormal-CT-Generation-MultiDisease/inference_full_ct_3d_with_body_mask.py b/Abnormal-CT-Generation-MultiDisease/inference_full_ct_3d_with_body_mask.py new file mode 100644 index 0000000000000000000000000000000000000000..5a42ac4e52d5ed498be137bb35bdd2b337b84da8 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/inference_full_ct_3d_with_body_mask.py @@ -0,0 +1,92 @@ +import glob +import nibabel as nib +import numpy as np +import os +import torch +from skimage import io +from einops import rearrange +from omegaconf import OmegaConf +from torch.utils.data import DataLoader +from tqdm import tqdm + +from ldm.util import instantiate_from_config + + +logdir = './logs/full_ct_3d_with_body_mask' +ckpt = os.path.join(logdir, "checkpoints", "epoch=000061.ckpt") + +configs_file = sorted(glob.glob(os.path.join(logdir, "configs/*.yaml")))[1] +configs = OmegaConf.load(configs_file) +model = instantiate_from_config(configs.model) +model.init_from_ckpt(ckpt) +model.eval() + +device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") +print("Using device", device) +model = model.to(device) + +config = OmegaConf.load('./configs/latent-diffusion/full_ct_3d_with_body_mask.yaml') +data = instantiate_from_config(config.data) +data.prepare_data() +data.setup() + +save_path = 'inference' +save_path = os.path.join(logdir, save_path) +if not os.path.exists(save_path): + os.makedirs(save_path) + +val_dataset = data.datasets['validation'] +batch_size = 1 +valloader = DataLoader(val_dataset, batch_size=batch_size, shuffle=False, num_workers=0, pin_memory=True) +val_num = len(val_dataset) +save_gt = True +# val_num = 10 +# breakpoint() +for idx, data in tqdm(enumerate(valloader)): + # if idx >= val_num: + # break + name=data['name'][0].split('.')[0] + volume_data = data['volume_data'] + volume_seg = data['volume_seg'] + input_text = data['input_text'] + # breakpoint() + + for z_slice in range(16): + slice_seg = volume_seg[:,:,z_slice:z_slice+16] + slice_ct = volume_data[:,:,z_slice:z_slice+16] + # print("shape:", slice_ct.shape) + # print("class:", torch.unique(slice_seg)) + # breakpoint() + input_data = {} + input_data['volume_data']=slice_ct.to(device) + input_data['volume_seg']=slice_seg.to(device) + input_data['input_text']=input_text + # breakpoint() + x, c = model.get_input(input_data, model.first_stage_key) + # breakpoint() + samples, _ = model.sample_log(cond=c, batch_size=16, ddim=True, eta=1., ddim_steps=200) + + res = model.decode_first_stage(samples) + res[res>1.0] = 1.0 + res[res<-1.0] = -1.0 + res = (res+1)/2 + # breakpoint() + res = res.mean(axis=1).detach().cpu().numpy() + # breakpoint() + slice_ct=(slice_ct+1)/2 + slice_seg=(slice_seg+1)/2 + # breakpoint() + for z_id in range(16): + input_img = np.clip(np.repeat(slice_ct[0,0,z_id][None,:].detach().cpu().numpy(), 3, 0),0,1) * 255 + res_ = np.repeat(res[z_id][None,:], 3, 0).transpose(1,2,0)* 255 + slice_seg_ = np.clip(np.repeat(slice_seg[0,0,z_id][None,:].detach().cpu().numpy(), 3, 0),0,1) * 255 + + io.imsave(os.path.join(save_path, str(name) + f'_sample_{z_id}.png'), res_.astype(np.uint8)) + io.imsave(os.path.join(save_path, str(name) + f'_input_img_{z_id}.png'), input_img.astype(np.uint8)) + io.imsave(os.path.join(save_path, str(name) + f'_seg_{z_id}.png'), slice_seg_.astype(np.uint8)) + + break + + break + + diff --git a/Abnormal-CT-Generation-MultiDisease/inference_train.json b/Abnormal-CT-Generation-MultiDisease/inference_train.json new file mode 100644 index 0000000000000000000000000000000000000000..ed3aaee445c3b2f1337730bdb12e2f004236f7f1 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/inference_train.json @@ -0,0 +1,1076 @@ +{"volume_path": "dataset/train_fixed/train_1741/train_1741_b/train_1741_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1741_b_2.nii.gz", "disease_mask": "seg_rxg/train_1741_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Irregularly circumscribed nodular consolidation area adjacent to the diaphragm in the basal segment of the lower lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1742/train_1742_c/train_1742_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1742_c_2.nii.gz", "disease_mask": "seg_rxg/train_1742_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Nodular lesions with ground glass opacity in the upper and middle lobes of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1742/train_1742_c/train_1742_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1742_c_2.nii.gz", "disease_mask": "seg_rxg/train_1742_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1742/train_1742_c/train_1742_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1742_c_2.nii.gz", "disease_mask": "seg_rxg/train_1742_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_1959/train_1959_b/train_1959_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1959_b_2.nii.gz", "disease_mask": "seg_rxg/train_1959_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Newly developed ground-glass opacities with increased density in the bilateral lower lobes of the lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1959/train_1959_b/train_1959_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1959_b_2.nii.gz", "disease_mask": "seg_rxg/train_1959_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Newly developed ground-glass opacities with increased density in the right middle lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1959/train_1959_b/train_1959_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1959_b_2.nii.gz", "disease_mask": "seg_rxg/train_1959_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Newly developed ground-glass opacities with increased density in the left lingular segments", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1959/train_1959_b/train_1959_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1959_b_2.nii.gz", "disease_mask": "seg_rxg/train_1959_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Occasional nodular consolidation in the bilateral lower lobes of the lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1959/train_1959_b/train_1959_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1959_b_2.nii.gz", "disease_mask": "seg_rxg/train_1959_b_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Occasional nodular consolidation in the right middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1959/train_1959_b/train_1959_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1959_b_2.nii.gz", "disease_mask": "seg_rxg/train_1959_b_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Atelectasis, consolidation", "disease_findings": "Occasional nodular consolidation in the left lingular segments", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1959/train_1959_b/train_1959_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1959_b_2.nii.gz", "disease_mask": "seg_rxg/train_1959_b_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Well-defined cavitary nodules measuring 8 mm in the superior segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1676/train_1676_a/train_1676_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1676_a_2.nii.gz", "disease_mask": "seg_rxg/train_1676_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Nodular ground-glass opacities in the peripheral subpleural regions of the lower lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1676/train_1676_a/train_1676_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1676_a_2.nii.gz", "disease_mask": "seg_rxg/train_1676_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Nodular ground-glass opacities in the middle lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2268/train_2268_a/train_2268_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2268_a_2.nii.gz", "disease_mask": "seg_rxg/train_2268_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the lower lobe of the left lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2268/train_2268_a/train_2268_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2268_a_2.nii.gz", "disease_mask": "seg_rxg/train_2268_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear atelectasis at the level of the left lung lower lobe fissure", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2268/train_2268_a/train_2268_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2268_a_2.nii.gz", "disease_mask": "seg_rxg/train_2268_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectasis in the lower lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1753/train_1753_e/train_1753_e_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1753_e_2.nii.gz", "disease_mask": "seg_rxg/train_1753_e_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Minimal effusion measuring approximately 8 mm at its deepest point in the right pleural space", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_1753/train_1753_e/train_1753_e_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1753_e_2.nii.gz", "disease_mask": "seg_rxg/train_1753_e_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Areas of increased density consistent with subsegmental-linear atelectasis in the lower lobes of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1753/train_1753_e/train_1753_e_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1753_e_2.nii.gz", "disease_mask": "seg_rxg/train_1753_e_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Areas of increased density consistent with subsegmental-linear atelectasis in the inferior lingular segment of the left upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1753/train_1753_e/train_1753_e_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1753_e_2.nii.gz", "disease_mask": "seg_rxg/train_1753_e_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Minimal ground glass density in adjacent areas", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1694/train_1694_a/train_1694_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1694_a_2.nii.gz", "disease_mask": "seg_rxg/train_1694_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Multiple calcified nodules measuring 2 mm at the posterobasal level of the right lower lobe", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_1694/train_1694_a/train_1694_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1694_a_2.nii.gz", "disease_mask": "seg_rxg/train_1694_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "5 mm calcified nodule at the anteromediobasal level of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1343/train_1343_a/train_1343_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1343_a_2.nii.gz", "disease_mask": "seg_rxg/train_1343_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Mild emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_1296/train_1296_a/train_1296_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1296_a_2.nii.gz", "disease_mask": "seg_rxg/train_1296_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Bilateral asymmetric diffuse pneumonic infiltration areas in the form of ground glass opacity in the upper lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1296/train_1296_a/train_1296_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1296_a_2.nii.gz", "disease_mask": "seg_rxg/train_1296_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular consolidation areas in the lower lobes", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2381/train_2381_a/train_2381_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2381_a_2.nii.gz", "disease_mask": "seg_rxg/train_2381_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific parenchymal nodules in the middle lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1441/train_1441_a/train_1441_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1441_a_2.nii.gz", "disease_mask": "seg_rxg/train_1441_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "4 mm non-contouring opacity with a minor fissure in the anterior segment of the upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1441/train_1441_a/train_1441_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1441_a_2.nii.gz", "disease_mask": "seg_rxg/train_1441_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectasis in the inferior segment of the left lung lingula", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1580/train_1580_a/train_1580_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1580_a_2.nii.gz", "disease_mask": "seg_rxg/train_1580_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "2 mm calcific nodule in the posterobasal region of the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1822/train_1822_b/train_1822_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1822_b_2.nii.gz", "disease_mask": "seg_rxg/train_1822_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral and centrally located ground glass opacities in the upper and lower lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1822/train_1822_b/train_1822_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1822_b_2.nii.gz", "disease_mask": "seg_rxg/train_1822_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral and centrally located ground glass opacities in the middle lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1822/train_1822_b/train_1822_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1822_b_2.nii.gz", "disease_mask": "seg_rxg/train_1822_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Round-shaped ground glass opacities", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1822/train_1822_b/train_1822_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1822_b_2.nii.gz", "disease_mask": "seg_rxg/train_1822_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities accompanied by nodules", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1822/train_1822_b/train_1822_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1822_b_2.nii.gz", "disease_mask": "seg_rxg/train_1822_b_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Findings consistent with viral pneumonia in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1384/train_1384_c/train_1384_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1384_c_2.nii.gz", "disease_mask": "seg_rxg/train_1384_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Density increases in the upper lobe and middle lobe anterior section of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1384/train_1384_c/train_1384_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1384_c_2.nii.gz", "disease_mask": "seg_rxg/train_1384_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Structural distortion in the upper lobe and middle lobe anterior section of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1384/train_1384_c/train_1384_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1384_c_2.nii.gz", "disease_mask": "seg_rxg/train_1384_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Volume loss in the upper lobe and middle lobe anterior section of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2250/train_2250_a/train_2250_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2250_a_2.nii.gz", "disease_mask": "seg_rxg/train_2250_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Faintly bordered parenchymal areas of light ground glass density in the basal segments of the lower lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2250/train_2250_a/train_2250_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2250_a_2.nii.gz", "disease_mask": "seg_rxg/train_2250_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the inferior segment of the lingula of the left upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1378/train_1378_a/train_1378_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1378_a_2.nii.gz", "disease_mask": "seg_rxg/train_1378_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the middle lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1378/train_1378_a/train_1378_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1378_a_2.nii.gz", "disease_mask": "seg_rxg/train_1378_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "4 mm subcentimeter nodules in the middle lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1378/train_1378_a/train_1378_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1378_a_2.nii.gz", "disease_mask": "seg_rxg/train_1378_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchiectasis", "disease_findings": "Bronchiectasis in the middle lobe of the right lung", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1378/train_1378_a/train_1378_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1378_a_2.nii.gz", "disease_mask": "seg_rxg/train_1378_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1396/train_1396_a/train_1396_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1396_a_2.nii.gz", "disease_mask": "seg_rxg/train_1396_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Area of increased density consistent with consolidation in the right lung upper lobe posterior segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1396/train_1396_a/train_1396_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1396_a_2.nii.gz", "disease_mask": "seg_rxg/train_1396_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Air bronchograms in the right lung upper lobe posterior segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1396/train_1396_a/train_1396_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1396_a_2.nii.gz", "disease_mask": "seg_rxg/train_1396_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Air bronchograms in the right lung lower lobe superior segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1396/train_1396_a/train_1396_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1396_a_2.nii.gz", "disease_mask": "seg_rxg/train_1396_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Vaguely circumscribed ground glass densities in the left lung upper lobe anterior segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1396/train_1396_a/train_1396_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1396_a_2.nii.gz", "disease_mask": "seg_rxg/train_1396_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Groundglass opacity", "disease_findings": "Vaguely circumscribed ground glass densities in the left lung apicoposterior segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1396/train_1396_a/train_1396_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1396_a_2.nii.gz", "disease_mask": "seg_rxg/train_1396_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Groundglass opacity", "disease_findings": "Vaguely circumscribed ground glass densities in the right lung middle lobe medial segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1396/train_1396_a/train_1396_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1396_a_2.nii.gz", "disease_mask": "seg_rxg/train_1396_a_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Groundglass opacity", "disease_findings": "Vaguely circumscribed ground glass densities in the right lung lower lobe superior segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1396/train_1396_a/train_1396_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1396_a_2.nii.gz", "disease_mask": "seg_rxg/train_1396_a_2.nii.gz", "disease_mask_channel": "7", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodules evaluated in favor of consolidation areas", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1396/train_1396_a/train_1396_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1396_a_2.nii.gz", "disease_mask": "seg_rxg/train_1396_a_2.nii.gz", "disease_mask_channel": "8", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lung parenchyma", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_2261/train_2261_a/train_2261_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2261_a_2.nii.gz", "disease_mask": "seg_rxg/train_2261_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground-glass consolidations forming a central-peripheral crazy paving pattern in the lower lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2261/train_2261_a/train_2261_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2261_a_2.nii.gz", "disease_mask": "seg_rxg/train_2261_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectatic changes in the right middle lobe and anterobasal segment of the right lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2261/train_2261_a/train_2261_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2261_a_2.nii.gz", "disease_mask": "seg_rxg/train_2261_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific subcentimeter parenchymal nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1313/train_1313_a/train_1313_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1313_a_2.nii.gz", "disease_mask": "seg_rxg/train_1313_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral nodular ground glass consolidations in all lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1313/train_1313_a/train_1313_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1313_a_2.nii.gz", "disease_mask": "seg_rxg/train_1313_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectatic changes in the left lung inferior lingular segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1313/train_1313_a/train_1313_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1313_a_2.nii.gz", "disease_mask": "seg_rxg/train_1313_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectatic changes in the right lung middle lobe medial segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1405/train_1405_a/train_1405_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1405_a_2.nii.gz", "disease_mask": "seg_rxg/train_1405_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectatic changes in the left lung lingular segment and lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1405/train_1405_a/train_1405_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1405_a_2.nii.gz", "disease_mask": "seg_rxg/train_1405_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Suspicious mass lesion in the posterior left lower lobe measuring 30x16 mm in axial sections", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1405/train_1405_a/train_1405_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1405_a_2.nii.gz", "disease_mask": "seg_rxg/train_1405_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pleural effusion or thickening", "disease_findings": "Effusion up to 13 mm in the deepest part of the left pleural space", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_1324/train_1324_a/train_1324_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1324_a_2.nii.gz", "disease_mask": "seg_rxg/train_1324_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Tubular bronchiectasis centrally in both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1324/train_1324_a/train_1324_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1324_a_2.nii.gz", "disease_mask": "seg_rxg/train_1324_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Centriacinar nodular infiltrates with ground glass density in basal and central regions of right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1324/train_1324_a/train_1324_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1324_a_2.nii.gz", "disease_mask": "seg_rxg/train_1324_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Septal thickening", "disease_findings": "Reticulonodular density increases in right lung apical and posterior segments", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_1324/train_1324_a/train_1324_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1324_a_2.nii.gz", "disease_mask": "seg_rxg/train_1324_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Septal thickening", "disease_findings": "Reticulonodular density increases in left lung upper lobe apicoposterior segment", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_1324/train_1324_a/train_1324_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1324_a_2.nii.gz", "disease_mask": "seg_rxg/train_1324_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Bronchiectasis", "disease_findings": "Traction bronchiectasis in right lung apical segment", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1435/train_1435_b/train_1435_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1435_b_2.nii.gz", "disease_mask": "seg_rxg/train_1435_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Paraseptal emphysematous changes in the upper lobes", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_1435/train_1435_b/train_1435_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1435_b_2.nii.gz", "disease_mask": "seg_rxg/train_1435_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchial wall thickening", "disease_findings": "Diffuse mild ectasia in the bronchial structures of both lungs, more prominent centrally", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_1435/train_1435_b/train_1435_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1435_b_2.nii.gz", "disease_mask": "seg_rxg/train_1435_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1486/train_1486_a/train_1486_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1486_a_2.nii.gz", "disease_mask": "seg_rxg/train_1486_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1592/train_1592_b/train_1592_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1592_b_2.nii.gz", "disease_mask": "seg_rxg/train_1592_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectasis in the central parts of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1592/train_1592_b/train_1592_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1592_b_2.nii.gz", "disease_mask": "seg_rxg/train_1592_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2257/train_2257_a/train_2257_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2257_a_2.nii.gz", "disease_mask": "seg_rxg/train_2257_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Areas of increased density consistent with linear atelectasis in the right lung middle lobe medial segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2257/train_2257_a/train_2257_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2257_a_2.nii.gz", "disease_mask": "seg_rxg/train_2257_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Areas of increased density consistent with linear atelectasis in the left lung upper lobe inferior lingular segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2257/train_2257_a/train_2257_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2257_a_2.nii.gz", "disease_mask": "seg_rxg/train_2257_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Areas of increased density consistent with linear atelectasis in the left lung lower lobe laterobasal segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2257/train_2257_a/train_2257_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2257_a_2.nii.gz", "disease_mask": "seg_rxg/train_2257_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Areas of increased density consistent with linear atelectasis in the left lung lower lobe posterobasal segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2257/train_2257_a/train_2257_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2257_a_2.nii.gz", "disease_mask": "seg_rxg/train_2257_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Parenchymal changes consistent with scarring", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2382/train_2382_a/train_2382_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2382_a_2.nii.gz", "disease_mask": "seg_rxg/train_2382_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground glass opacities in the laterobasal and posterobasal sections of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2123/train_2123_a/train_2123_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2123_a_2.nii.gz", "disease_mask": "seg_rxg/train_2123_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "5.5 mm ground-glass opacity in the anterior segment of the left upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2010/train_2010_a/train_2010_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2010_a_2.nii.gz", "disease_mask": "seg_rxg/train_2010_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Fibrotic retraction with associated cicatricial bronchiectasis and calcifications in apical and anterior segments of right upper lobe", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_2010/train_2010_a/train_2010_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2010_a_2.nii.gz", "disease_mask": "seg_rxg/train_2010_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Minimal fibrotic changes in medial segment of right middle lobe", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2010/train_2010_a/train_2010_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2010_a_2.nii.gz", "disease_mask": "seg_rxg/train_2010_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectatic changes in medial segment of right middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2010/train_2010_a/train_2010_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2010_a_2.nii.gz", "disease_mask": "seg_rxg/train_2010_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Micronodules", "disease_findings": "Minimal fibrotic changes in inferior lingular segment of left lung", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2010/train_2010_a/train_2010_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2010_a_2.nii.gz", "disease_mask": "seg_rxg/train_2010_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectatic changes in inferior lingular segment of left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2010/train_2010_a/train_2010_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2010_a_2.nii.gz", "disease_mask": "seg_rxg/train_2010_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Micronodules", "disease_findings": "Calcific nodules measuring 2.6 mm in anterior segment of left upper lobe", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2010/train_2010_a/train_2010_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2010_a_2.nii.gz", "disease_mask": "seg_rxg/train_2010_a_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Micronodules", "disease_findings": "Calcific nodules in basal part of right middle lobe", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2010/train_2010_a/train_2010_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2010_a_2.nii.gz", "disease_mask": "seg_rxg/train_2010_a_2.nii.gz", "disease_mask_channel": "7", "disease_label": "Micronodules", "disease_findings": "Calcific nodules in superior segment of right lower lobe", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_1653/train_1653_a/train_1653_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1653_a_2.nii.gz", "disease_mask": "seg_rxg/train_1653_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral localized ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1758/train_1758_b/train_1758_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1758_b_2.nii.gz", "disease_mask": "seg_rxg/train_1758_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Significant consolidated densities with a halo sign in the lung parenchyma", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1758/train_1758_b/train_1758_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1758_b_2.nii.gz", "disease_mask": "seg_rxg/train_1758_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Variable changes in consolidated densities with some decreasing in size, some increasing, and some new lesions", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1758/train_1758_b/train_1758_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1758_b_2.nii.gz", "disease_mask": "seg_rxg/train_1758_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "New lesion measuring up to 23 mm in the paracardiac paramediastinal area, particularly in the anterior upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1758/train_1758_b/train_1758_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1758_b_2.nii.gz", "disease_mask": "seg_rxg/train_1758_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Previously described consolidated areas in the left upper lobe apicoposterior region have shrunk", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1758/train_1758_b/train_1758_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1758_b_2.nii.gz", "disease_mask": "seg_rxg/train_1758_b_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Cavitation in the left upper lobe apicoposterior region", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2139/train_2139_c/train_2139_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2139_c_2.nii.gz", "disease_mask": "seg_rxg/train_2139_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Fibrotic densities in the apical segments of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2139/train_2139_c/train_2139_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2139_c_2.nii.gz", "disease_mask": "seg_rxg/train_2139_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground-glass opacities in the lower lobes of both lungs, predominantly in the subpleural areas", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1313/train_1313_b/train_1313_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1313_b_2.nii.gz", "disease_mask": "seg_rxg/train_1313_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass areas in the posterior subpleural area of the lower lobes of both lungs, more prominently in the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1313/train_1313_b/train_1313_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1313_b_2.nii.gz", "disease_mask": "seg_rxg/train_1313_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Minimal ground-glass areas in the peripheral regions of both lungs in the upper lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1868/train_1868_a/train_1868_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1868_a_2.nii.gz", "disease_mask": "seg_rxg/train_1868_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Structural distortion in the apical segments of lung parenchyma, more prominent on the left", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1868/train_1868_a/train_1868_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1868_a_2.nii.gz", "disease_mask": "seg_rxg/train_1868_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Macrocalcified nodular lesions in both lung parenchyma", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1868/train_1868_a/train_1868_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1868_a_2.nii.gz", "disease_mask": "seg_rxg/train_1868_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Findings consistent with TB sequelae in the apices of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1868/train_1868_a/train_1868_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1868_a_2.nii.gz", "disease_mask": "seg_rxg/train_1868_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Indistinct ground-glass density appearance in the posterior segment of the right upper lobe, adjacent to the bronchovascular structure in the peripheral area", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2100/train_2100_a/train_2100_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2100_a_2.nii.gz", "disease_mask": "seg_rxg/train_2100_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Fibrotic density increases with reticulonodular changes in both lung apices", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_2100/train_2100_a/train_2100_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2100_a_2.nii.gz", "disease_mask": "seg_rxg/train_2100_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchiectasis", "disease_findings": "Segmental tubular bronchiectasis in both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_2100/train_2100_a/train_2100_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2100_a_2.nii.gz", "disease_mask": "seg_rxg/train_2100_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific parenchymal nodules in left lower lobe anteromediobasal segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2100/train_2100_a/train_2100_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2100_a_2.nii.gz", "disease_mask": "seg_rxg/train_2100_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific parenchymal nodules in left upper lobe lingular segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2100/train_2100_a/train_2100_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2100_a_2.nii.gz", "disease_mask": "seg_rxg/train_2100_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific parenchymal nodules in right middle lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1837/train_1837_a/train_1837_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1837_a_2.nii.gz", "disease_mask": "seg_rxg/train_1837_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Patchy consolidation areas with air bronchograms in subpleural locations in all segments of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1837/train_1837_a/train_1837_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1837_a_2.nii.gz", "disease_mask": "seg_rxg/train_1837_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities adjacent to patchy consolidation areas in all segments of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1790/train_1790_a/train_1790_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1790_a_2.nii.gz", "disease_mask": "seg_rxg/train_1790_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Multilobar, mostly peripheral subpleural nodular consolidation in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1790/train_1790_a/train_1790_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1790_a_2.nii.gz", "disease_mask": "seg_rxg/train_1790_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1610/train_1610_a/train_1610_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1610_a_2.nii.gz", "disease_mask": "seg_rxg/train_1610_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific parenchymal nodules in the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1610/train_1610_a/train_1610_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1610_a_2.nii.gz", "disease_mask": "seg_rxg/train_1610_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest nodule measuring 5 x 5.8 mm in the subpleural region of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2330/train_2330_a/train_2330_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2330_a_2.nii.gz", "disease_mask": "seg_rxg/train_2330_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Widespread calcified pleural plaques with a tendency to coalesce in both lungs", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_2330/train_2330_a/train_2330_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2330_a_2.nii.gz", "disease_mask": "seg_rxg/train_2330_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Scarring in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1504/train_1504_b/train_1504_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1504_b_2.nii.gz", "disease_mask": "seg_rxg/train_1504_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Honeycombing", "disease_findings": "Honeycomb appearance indicative of fibrosis in the upper lobes of both lungs, especially in the peripheral areas", "disease_class": 155, "disease_label_text": "Honeycombing"} +{"volume_path": "dataset/train_fixed/train_1504/train_1504_b/train_1504_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1504_b_2.nii.gz", "disease_mask": "seg_rxg/train_1504_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_1504/train_1504_b/train_1504_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1504_b_2.nii.gz", "disease_mask": "seg_rxg/train_1504_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Minimal ground glass areas in the central and peripheral parts of the upper lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1504/train_1504_b/train_1504_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1504_b_2.nii.gz", "disease_mask": "seg_rxg/train_1504_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Mass in the medial aspect of the anterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2399/train_2399_a/train_2399_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2399_a_2.nii.gz", "disease_mask": "seg_rxg/train_2399_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities in the peripheral regions of the upper and lower lobes of both lungs and the middle lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2399/train_2399_a/train_2399_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2399_a_2.nii.gz", "disease_mask": "seg_rxg/train_2399_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidations in the peripheral regions of the upper and lower lobes of both lungs and the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2399/train_2399_a/train_2399_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2399_a_2.nii.gz", "disease_mask": "seg_rxg/train_2399_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear densities in the peripheral regions of the upper and lower lobes of both lungs and the middle lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1758/train_1758_a/train_1758_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1758_a_2.nii.gz", "disease_mask": "seg_rxg/train_1758_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple focal nodular lesions with spiculated contours in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1758/train_1758_a/train_1758_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1758_a_2.nii.gz", "disease_mask": "seg_rxg/train_1758_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Spiculated nodule in the posterior segment of the left lower lobe, measuring up to 1.8 cm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1758/train_1758_a/train_1758_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1758_a_2.nii.gz", "disease_mask": "seg_rxg/train_1758_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Halo sign in the spiculated nodule in the posterior segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1348/train_1348_g/train_1348_g_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1348_g_2.nii.gz", "disease_mask": "seg_rxg/train_1348_g_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Bronchiectasis in areas of ground glass opacity", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1348/train_1348_g/train_1348_g_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1348_g_2.nii.gz", "disease_mask": "seg_rxg/train_1348_g_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Cavities in areas of ground glass opacity", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1348/train_1348_g/train_1348_g_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1348_g_2.nii.gz", "disease_mask": "seg_rxg/train_1348_g_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Increased cavitation in areas of consolidation in the right upper lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1348/train_1348_g/train_1348_g_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1348_g_2.nii.gz", "disease_mask": "seg_rxg/train_1348_g_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Increased cavitation in areas of consolidation in the right lower lobe posterobasal segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1348/train_1348_g/train_1348_g_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1348_g_2.nii.gz", "disease_mask": "seg_rxg/train_1348_g_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Stable nodular infiltrates in the left lower lobe mediobasal segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1348/train_1348_g/train_1348_g_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1348_g_2.nii.gz", "disease_mask": "seg_rxg/train_1348_g_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Bronchiectasis", "disease_findings": "Bronchiectasis in the left lower lobe mediobasal segment", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1348/train_1348_g/train_1348_g_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1348_g_2.nii.gz", "disease_mask": "seg_rxg/train_1348_g_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Bronchial wall thickening", "disease_findings": "Bronchial wall thickening in the left lower lobe mediobasal segment", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_1348/train_1348_g/train_1348_g_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1348_g_2.nii.gz", "disease_mask": "seg_rxg/train_1348_g_2.nii.gz", "disease_mask_channel": "7", "disease_label": "Micronodules", "disease_findings": "Newly developing tree-in-bud patterns in the left lung superior lingular segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_1348/train_1348_g/train_1348_g_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1348_g_2.nii.gz", "disease_mask": "seg_rxg/train_1348_g_2.nii.gz", "disease_mask_channel": "8", "disease_label": "Micronodules", "disease_findings": "Newly developing tree-in-bud patterns in the right middle lobe lateral segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_1636/train_1636_a/train_1636_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1636_a_2.nii.gz", "disease_mask": "seg_rxg/train_1636_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "8 mm ground glass opacity adjacent to the fissure in the anterior lower lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2009/train_2009_a/train_2009_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2009_a_2.nii.gz", "disease_mask": "seg_rxg/train_2009_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subpleural nodule approximately 5 mm in diameter in the lateral segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2009/train_2009_a/train_2009_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2009_a_2.nii.gz", "disease_mask": "seg_rxg/train_2009_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the lateral part of the left lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2009/train_2009_a/train_2009_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2009_a_2.nii.gz", "disease_mask": "seg_rxg/train_2009_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the left upper lobe inferior lingular segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2009/train_2009_a/train_2009_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2009_a_2.nii.gz", "disease_mask": "seg_rxg/train_2009_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the middle lobe medial segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2009/train_2009_a/train_2009_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2009_a_2.nii.gz", "disease_mask": "seg_rxg/train_2009_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several additional subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2029/train_2029_a/train_2029_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2029_a_2.nii.gz", "disease_mask": "seg_rxg/train_2029_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear subsegmental atelectasis in the lower lobes of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2029/train_2029_a/train_2029_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2029_a_2.nii.gz", "disease_mask": "seg_rxg/train_2029_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchial wall thickening", "disease_findings": "Peribronchovascular thickening in the lower lobes of both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_2029/train_2029_a/train_2029_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2029_a_2.nii.gz", "disease_mask": "seg_rxg/train_2029_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Nonspecific ground-glass opacities in the apicoposterior segment of the left upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2029/train_2029_a/train_2029_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2029_a_2.nii.gz", "disease_mask": "seg_rxg/train_2029_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Nonspecific ground-glass opacities in the superior and inferior lingular segments of the left upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2029/train_2029_a/train_2029_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2029_a_2.nii.gz", "disease_mask": "seg_rxg/train_2029_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific subcentimeter pulmonary nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1305/train_1305_a/train_1305_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1305_a_2.nii.gz", "disease_mask": "seg_rxg/train_1305_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Dependent density increases in the lower lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2002/train_2002_a/train_2002_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2002_a_2.nii.gz", "disease_mask": "seg_rxg/train_2002_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass opacities in a few localized areas in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2002/train_2002_a/train_2002_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2002_a_2.nii.gz", "disease_mask": "seg_rxg/train_2002_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Scarring changes at the apices", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2002/train_2002_a/train_2002_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2002_a_2.nii.gz", "disease_mask": "seg_rxg/train_2002_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "2 mm nodule in the laterobasal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1650/train_1650_a/train_1650_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1650_a_2.nii.gz", "disease_mask": "seg_rxg/train_1650_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific 2.5 mm nodule in the superior segment of the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2385/train_2385_a/train_2385_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2385_a_2.nii.gz", "disease_mask": "seg_rxg/train_2385_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground glass opacities in the posterior and lateral segments of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2139/train_2139_b/train_2139_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2139_b_2.nii.gz", "disease_mask": "seg_rxg/train_2139_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2139/train_2139_b/train_2139_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2139_b_2.nii.gz", "disease_mask": "seg_rxg/train_2139_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectasis in both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_2139/train_2139_b/train_2139_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2139_b_2.nii.gz", "disease_mask": "seg_rxg/train_2139_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Pleuroparenchymal scarring changes in the apices of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1856/train_1856_a/train_1856_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1856_a_2.nii.gz", "disease_mask": "seg_rxg/train_1856_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1794/train_1794_a/train_1794_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1794_a_2.nii.gz", "disease_mask": "seg_rxg/train_1794_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Diffusely located peribronchial and subpleural ground-glass opacities with faint borders in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1794/train_1794_a/train_1794_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1794_a_2.nii.gz", "disease_mask": "seg_rxg/train_1794_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Two subpleural nodules in the superior lower lobe of the right lung, largest measuring 4.5 mm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2078/train_2078_a/train_2078_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2078_a_2.nii.gz", "disease_mask": "seg_rxg/train_2078_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in the lower lobe of the left lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1796/train_1796_a/train_1796_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1796_a_2.nii.gz", "disease_mask": "seg_rxg/train_1796_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the inferior subsegment of the lingular segment of the left upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1796/train_1796_a/train_1796_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1796_a_2.nii.gz", "disease_mask": "seg_rxg/train_1796_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Subcentimeter calcific nodules in both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_1301/train_1301_a/train_1301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1301_a_2.nii.gz", "disease_mask": "seg_rxg/train_1301_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Scarring in the posterior costal pleura of the right hemithorax", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1301/train_1301_a/train_1301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1301_a_2.nii.gz", "disease_mask": "seg_rxg/train_1301_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchiectasis", "disease_findings": "Segmental and subsegmental tubular bronchiectasis in both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1301/train_1301_a/train_1301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1301_a_2.nii.gz", "disease_mask": "seg_rxg/train_1301_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchial wall thickening", "disease_findings": "Peribronchial thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_1301/train_1301_a/train_1301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1301_a_2.nii.gz", "disease_mask": "seg_rxg/train_1301_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Central-peripheral crazy paving pattern with small, patchy, faintly limited ground-glass opacities in the left lung inferior lingular segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1301/train_1301_a/train_1301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1301_a_2.nii.gz", "disease_mask": "seg_rxg/train_1301_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Groundglass opacity", "disease_findings": "Central-peripheral crazy paving pattern with small, patchy, faintly limited ground-glass opacities in the left lung lower lobe basal segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1301/train_1301_a/train_1301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1301_a_2.nii.gz", "disease_mask": "seg_rxg/train_1301_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Groundglass opacity", "disease_findings": "Central-peripheral crazy paving pattern with small, patchy, faintly limited ground-glass opacities in the right lung middle lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1301/train_1301_a/train_1301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1301_a_2.nii.gz", "disease_mask": "seg_rxg/train_1301_a_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Atelectasis, consolidation", "disease_findings": "Scarring and atelectasis in the right lung middle lobe medial segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1301/train_1301_a/train_1301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1301_a_2.nii.gz", "disease_mask": "seg_rxg/train_1301_a_2.nii.gz", "disease_mask_channel": "7", "disease_label": "Atelectasis, consolidation", "disease_findings": "Scarring and atelectasis in the left lung upper lobe inferior lingular segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1301/train_1301_a/train_1301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1301_a_2.nii.gz", "disease_mask": "seg_rxg/train_1301_a_2.nii.gz", "disease_mask_channel": "8", "disease_label": "Atelectasis, consolidation", "disease_findings": "Scarring and atelectasis in both lung lower lobe basal segments", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1301/train_1301_a/train_1301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1301_a_2.nii.gz", "disease_mask": "seg_rxg/train_1301_a_2.nii.gz", "disease_mask_channel": "9", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter nonspecific pulmonary nodules in the lung parenchyma, stable", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1299/train_1299_a/train_1299_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1299_a_2.nii.gz", "disease_mask": "seg_rxg/train_1299_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchiectasis", "disease_findings": "Bronchiectasis in both lungs, especially in the lower lobes and central parts", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1299/train_1299_a/train_1299_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1299_a_2.nii.gz", "disease_mask": "seg_rxg/train_1299_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation in the posterior part of the lower lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1299/train_1299_a/train_1299_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1299_a_2.nii.gz", "disease_mask": "seg_rxg/train_1299_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Micronodules", "disease_findings": "Tree-in-bud appearances, more prominent in the lower lobes and posterior part of both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_1299/train_1299_a/train_1299_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1299_a_2.nii.gz", "disease_mask": "seg_rxg/train_1299_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular consolidation in a small area in the anterior segment of the upper lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2402/train_2402_a/train_2402_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2402_a_2.nii.gz", "disease_mask": "seg_rxg/train_2402_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "5.3 mm nodule in the lingula inferior segment of the left lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1570/train_1570_b/train_1570_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1570_b_2.nii.gz", "disease_mask": "seg_rxg/train_1570_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidations in the upper lobes and peripheral areas of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1570/train_1570_b/train_1570_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1570_b_2.nii.gz", "disease_mask": "seg_rxg/train_1570_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear density increases in the upper lobes and peripheral areas of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1570/train_1570_b/train_1570_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1570_b_2.nii.gz", "disease_mask": "seg_rxg/train_1570_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Regions of ground glass opacity in the upper lobes and peripheral areas of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1976/train_1976_a/train_1976_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1976_a_2.nii.gz", "disease_mask": "seg_rxg/train_1976_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Areas of increased density consistent with scarring in the apices of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1976/train_1976_a/train_1976_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1976_a_2.nii.gz", "disease_mask": "seg_rxg/train_1976_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchiectasis", "disease_findings": "Mild bronchiectatic changes with central prominence in both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1976/train_1976_a/train_1976_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1976_a_2.nii.gz", "disease_mask": "seg_rxg/train_1976_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Air cyst measuring 1.5 cm in diameter in the posterobasal segment of the lower lobe of the left lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1742/train_1742_b/train_1742_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1742_b_2.nii.gz", "disease_mask": "seg_rxg/train_1742_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Total resorption in the pleural effusion previously noted in the left hemithorax", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_1742/train_1742_b/train_1742_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1742_b_2.nii.gz", "disease_mask": "seg_rxg/train_1742_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Complete resolution of ground-glass opacities in the left lung lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2201/train_2201_a/train_2201_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2201_a_2.nii.gz", "disease_mask": "seg_rxg/train_2201_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Subcentimeter centracinar nodules in the peripheral areas of the basal segments of the lower lobes of both lungs, more prominently on the right", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_1891/train_1891_a/train_1891_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1891_a_2.nii.gz", "disease_mask": "seg_rxg/train_1891_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "7.2x5 mm intrapulmonary nodule in the anterior segment of the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1448/train_1448_c/train_1448_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1448_c_2.nii.gz", "disease_mask": "seg_rxg/train_1448_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Diffuse emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_1448/train_1448_c/train_1448_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1448_c_2.nii.gz", "disease_mask": "seg_rxg/train_1448_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis adjacent to the effusion in both lower lobes", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1448/train_1448_c/train_1448_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1448_c_2.nii.gz", "disease_mask": "seg_rxg/train_1448_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Large consolidation-mass with a cavity in the central part of the right upper lobe, measuring 8.5 cm at its widest point", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1448/train_1448_c/train_1448_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1448_c_2.nii.gz", "disease_mask": "seg_rxg/train_1448_c_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Thick-walled lesion in the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1448/train_1448_c/train_1448_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1448_c_2.nii.gz", "disease_mask": "seg_rxg/train_1448_c_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Micronodules", "disease_findings": "Budding tree appearances in the right lung middle lobe and lower lobe adjacent to the described area", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_1448/train_1448_c/train_1448_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1448_c_2.nii.gz", "disease_mask": "seg_rxg/train_1448_c_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Micronodules", "disease_findings": "Budding tree appearances in a small area of the left lower lobe", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_1269/train_1269_a/train_1269_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1269_a_2.nii.gz", "disease_mask": "seg_rxg/train_1269_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral and central ground-glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1269/train_1269_a/train_1269_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1269_a_2.nii.gz", "disease_mask": "seg_rxg/train_1269_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Findings more pronounced in the peripheral regions", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1269/train_1269_a/train_1269_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1269_a_2.nii.gz", "disease_mask": "seg_rxg/train_1269_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Septal thickening", "disease_findings": "Minimal interlobular septal thickening within the ground-glass areas", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_2241/train_2241_a/train_2241_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2241_a_2.nii.gz", "disease_mask": "seg_rxg/train_2241_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific nodules in both lungs, largest measuring 4.5 mm in diameter in the anterior segment of the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2241/train_2241_a/train_2241_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2241_a_2.nii.gz", "disease_mask": "seg_rxg/train_2241_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass parenchymal nodule approximately 5 mm in diameter at the level of the major fissure in the lateral segment of the right middle lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1511/train_1511_a/train_1511_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1511_a_2.nii.gz", "disease_mask": "seg_rxg/train_1511_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear and nodular density increases in the laterobasal segment of the left lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1511/train_1511_a/train_1511_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1511_a_2.nii.gz", "disease_mask": "seg_rxg/train_1511_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Minimal structural distortion in the laterobasal segment of the left lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1511/train_1511_a/train_1511_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1511_a_2.nii.gz", "disease_mask": "seg_rxg/train_1511_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Minimal volume loss in the laterobasal segment of the left lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1511/train_1511_a/train_1511_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1511_a_2.nii.gz", "disease_mask": "seg_rxg/train_1511_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_1511/train_1511_a/train_1511_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1511_a_2.nii.gz", "disease_mask": "seg_rxg/train_1511_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pleural effusion or thickening", "disease_findings": "Bilateral minimal pleural effusion", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_2352/train_2352_a/train_2352_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2352_a_2.nii.gz", "disease_mask": "seg_rxg/train_2352_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few nonspecific subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2993/train_2993_a/train_2993_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2993_a_2.nii.gz", "disease_mask": "seg_rxg/train_2993_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Interlobar and intralobular septal thickening in both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_2993/train_2993_a/train_2993_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2993_a_2.nii.gz", "disease_mask": "seg_rxg/train_2993_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Extensive areas of nodular consolidation in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2993/train_2993_a/train_2993_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2993_a_2.nii.gz", "disease_mask": "seg_rxg/train_2993_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Centriacinar nodular infiltrates in the lower lobe of the left lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2877/train_2877_a/train_2877_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2877_a_2.nii.gz", "disease_mask": "seg_rxg/train_2877_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the middle lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_3398/train_3398_b/train_3398_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3398_b_2.nii.gz", "disease_mask": "seg_rxg/train_3398_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Diffuse centrilobular emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_3398/train_3398_b/train_3398_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3398_b_2.nii.gz", "disease_mask": "seg_rxg/train_3398_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Centriacinar subcentimeter nodules in both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_3313/train_3313_a/train_3313_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3313_a_2.nii.gz", "disease_mask": "seg_rxg/train_3313_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific nodules in both lungs, largest measuring 5 mm in the lingular segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2918/train_2918_a/train_2918_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2918_a_2.nii.gz", "disease_mask": "seg_rxg/train_2918_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules in both lungs, largest measuring approximately 4 mm in diameter", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2977/train_2977_a/train_2977_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2977_a_2.nii.gz", "disease_mask": "seg_rxg/train_2977_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Widespread subpleural ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2977/train_2977_a/train_2977_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2977_a_2.nii.gz", "disease_mask": "seg_rxg/train_2977_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidations in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2977/train_2977_a/train_2977_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2977_a_2.nii.gz", "disease_mask": "seg_rxg/train_2977_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchial wall thickening", "disease_findings": "Focal bronchial enlargement within areas of consolidation", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_2899/train_2899_a/train_2899_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2899_a_2.nii.gz", "disease_mask": "seg_rxg/train_2899_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Peribronchial sheath thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_2899/train_2899_a/train_2899_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2899_a_2.nii.gz", "disease_mask": "seg_rxg/train_2899_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2899/train_2899_a/train_2899_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2899_a_2.nii.gz", "disease_mask": "seg_rxg/train_2899_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion measuring 28 mm on the right", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_2899/train_2899_a/train_2899_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2899_a_2.nii.gz", "disease_mask": "seg_rxg/train_2899_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion measuring 17 mm on the left", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_2806/train_2806_a/train_2806_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2806_a_2.nii.gz", "disease_mask": "seg_rxg/train_2806_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Solitary nodule measuring 7.6 x 6 mm with slightly irregular borders and extending to the spicule in the apicoposterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2806/train_2806_a/train_2806_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2806_a_2.nii.gz", "disease_mask": "seg_rxg/train_2806_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Nonspecific calcific nodules in the right lung", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2806/train_2806_a/train_2806_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2806_a_2.nii.gz", "disease_mask": "seg_rxg/train_2806_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Pleuroparenchymal fibroatelectatic changes causing parenchymal distortion and minimal volume loss in the right middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2806/train_2806_a/train_2806_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2806_a_2.nii.gz", "disease_mask": "seg_rxg/train_2806_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Pleuroparenchymal fibroatelectatic changes causing parenchymal distortion and minimal volume loss in the left upper lobe inferior lingular segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2806/train_2806_a/train_2806_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2806_a_2.nii.gz", "disease_mask": "seg_rxg/train_2806_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Pleuroparenchymal fibroatelectatic changes causing parenchymal distortion and minimal volume loss in both lower lobe posterobasal segments", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2806/train_2806_a/train_2806_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2806_a_2.nii.gz", "disease_mask": "seg_rxg/train_2806_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Atelectasis, consolidation", "disease_findings": "Pleuroparenchymal fibroatelectatic changes causing parenchymal distortion and minimal volume loss in the right lower lobe mediobasal segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_b/train_2718_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_b_2.nii.gz", "disease_mask": "seg_rxg/train_2718_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Findings suggestive of scarring in both lung apexes", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_b/train_2718_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_b_2.nii.gz", "disease_mask": "seg_rxg/train_2718_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear atelectasis in both lower lobes", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_b/train_2718_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_b_2.nii.gz", "disease_mask": "seg_rxg/train_2718_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few subcentimeter nonspecific nodules in the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2974/train_2974_a/train_2974_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2974_a_2.nii.gz", "disease_mask": "seg_rxg/train_2974_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular consolidation in the posterobasal segment of the lower lobe of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2974/train_2974_a/train_2974_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2974_a_2.nii.gz", "disease_mask": "seg_rxg/train_2974_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Peripherally located interlobular septal thickenings in the posterobasal segment of the lower lobe of the left lung", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_2974/train_2974_a/train_2974_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2974_a_2.nii.gz", "disease_mask": "seg_rxg/train_2974_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Surrounding ground-glass opacities in the posterobasal segment of the lower lobe of the left lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2974/train_2974_a/train_2974_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2974_a_2.nii.gz", "disease_mask": "seg_rxg/train_2974_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific parenchymal nodule in the anterobasal segment of the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2974/train_2974_a/train_2974_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2974_a_2.nii.gz", "disease_mask": "seg_rxg/train_2974_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Bronchial wall thickening", "disease_findings": "Segmental and subsegmental peribronchial thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_2719/train_2719_b/train_2719_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2719_b_2.nii.gz", "disease_mask": "seg_rxg/train_2719_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Increased density and minimal structural distortion in the posterior part of the upper lobe apical segment of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2719/train_2719_b/train_2719_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2719_b_2.nii.gz", "disease_mask": "seg_rxg/train_2719_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Suggestive of scarring in the posterior part of the upper lobe apical segment of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2719/train_2719_b/train_2719_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2719_b_2.nii.gz", "disease_mask": "seg_rxg/train_2719_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Minimal pleuroparenchymal scarring in the left lung apex", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2719/train_2719_b/train_2719_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2719_b_2.nii.gz", "disease_mask": "seg_rxg/train_2719_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2719/train_2719_b/train_2719_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2719_b_2.nii.gz", "disease_mask": "seg_rxg/train_2719_b_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_2719/train_2719_b/train_2719_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2719_b_2.nii.gz", "disease_mask": "seg_rxg/train_2719_b_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Pleural effusion or thickening", "disease_findings": "Bilateral minimal pleural effusion, more prominent on the right", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_2719/train_2719_b/train_2719_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2719_b_2.nii.gz", "disease_mask": "seg_rxg/train_2719_b_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the basal segments of the lower lobe of the lung adjacent to the pleural effusion", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2775/train_2775_a/train_2775_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2775_a_2.nii.gz", "disease_mask": "seg_rxg/train_2775_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter nonspecific parenchymal nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2965/train_2965_b/train_2965_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2965_b_2.nii.gz", "disease_mask": "seg_rxg/train_2965_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Right pleural effusion measuring 1.5 cm at its deepest part", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_2965/train_2965_b/train_2965_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2965_b_2.nii.gz", "disease_mask": "seg_rxg/train_2965_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Left pleural effusion measuring 1.0 cm at its deepest part", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_2965/train_2965_b/train_2965_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2965_b_2.nii.gz", "disease_mask": "seg_rxg/train_2965_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Diffuse subpleural and intraparenchymal mass lesions in all segments of both lung parenchyma", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2965/train_2965_b/train_2965_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2965_b_2.nii.gz", "disease_mask": "seg_rxg/train_2965_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Metastatic mass lesions in right lower lobe superior, lower lobe mediobasal, left upper lobe posterior, and left lower lobe superior segments", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2965/train_2965_b/train_2965_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2965_b_2.nii.gz", "disease_mask": "seg_rxg/train_2965_b_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodules resembling a budding tree pattern", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2965/train_2965_b/train_2965_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2965_b_2.nii.gz", "disease_mask": "seg_rxg/train_2965_b_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Endobronchial spread", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2965/train_2965_b/train_2965_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2965_b_2.nii.gz", "disease_mask": "seg_rxg/train_2965_b_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Newly developed area measuring approximately 2.0x1.3 cm with air bronchograms in the paramediastinal area of the left lung lingula superior segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_3770/train_3770_a/train_3770_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3770_a_2.nii.gz", "disease_mask": "seg_rxg/train_3770_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Peripheral predominantly diffuse consolidations in almost all lobes, particularly in the lower lobes and posterior regions of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_3770/train_3770_a/train_3770_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3770_a_2.nii.gz", "disease_mask": "seg_rxg/train_3770_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in almost all lobes, particularly in the lower lobes and posterior regions of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_3651/train_3651_a/train_3651_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3651_a_2.nii.gz", "disease_mask": "seg_rxg/train_3651_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Multiple ground glass opacities in both lungs with a halo sign surrounding the nodules in a patchy distribution", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2818/train_2818_a/train_2818_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2818_a_2.nii.gz", "disease_mask": "seg_rxg/train_2818_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Peripheral and central consolidations in the upper and lower lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2818/train_2818_a/train_2818_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2818_a_2.nii.gz", "disease_mask": "seg_rxg/train_2818_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Peripheral and central consolidations in the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2818/train_2818_a/train_2818_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2818_a_2.nii.gz", "disease_mask": "seg_rxg/train_2818_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Minimal ground glass opacities in the upper and lower lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2986/train_2986_c/train_2986_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2986_c_2.nii.gz", "disease_mask": "seg_rxg/train_2986_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Centriacinar pulmonary nodules of ground glass density in both lungs, particularly in the upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2986/train_2986_c/train_2986_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2986_c_2.nii.gz", "disease_mask": "seg_rxg/train_2986_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Interlobular septal thickening in the lower lobe of the left lung and the lower lobes of the right lung", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_2986/train_2986_c/train_2986_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2986_c_2.nii.gz", "disease_mask": "seg_rxg/train_2986_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pleural effusion or thickening", "disease_findings": "Minimal pleural effusion in the posterobasal segment of the left lower lobe", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_2986/train_2986_c/train_2986_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2986_c_2.nii.gz", "disease_mask": "seg_rxg/train_2986_c_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the posterobasal segment of the left lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_3640/train_3640_a/train_3640_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3640_a_2.nii.gz", "disease_mask": "seg_rxg/train_3640_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Air cysts in both lungs, largest measuring approximately 1 cm in diameter", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_3576/train_3576_a/train_3576_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3576_a_2.nii.gz", "disease_mask": "seg_rxg/train_3576_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2867/train_2867_a/train_2867_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2867_a_2.nii.gz", "disease_mask": "seg_rxg/train_2867_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Pleuroparenchymal scarring changes in both lung apices", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2867/train_2867_a/train_2867_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2867_a_2.nii.gz", "disease_mask": "seg_rxg/train_2867_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchial wall thickening", "disease_findings": "Minimal peribronchial thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_2867/train_2867_a/train_2867_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2867_a_2.nii.gz", "disease_mask": "seg_rxg/train_2867_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2888/train_2888_a/train_2888_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2888_a_2.nii.gz", "disease_mask": "seg_rxg/train_2888_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule in the right lung middle lobe near the fissure, measuring 5.8x6.8 mm with irregular contours", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2888/train_2888_a/train_2888_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2888_a_2.nii.gz", "disease_mask": "seg_rxg/train_2888_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "3 mm nodule in the right lung middle lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2888/train_2888_a/train_2888_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2888_a_2.nii.gz", "disease_mask": "seg_rxg/train_2888_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Pleuroparenchymal scarring in the posterobasal segment of the left lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_3709/train_3709_a/train_3709_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3709_a_2.nii.gz", "disease_mask": "seg_rxg/train_3709_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Fibroatelectatic changes in the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_3709/train_3709_a/train_3709_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3709_a_2.nii.gz", "disease_mask": "seg_rxg/train_3709_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchial wall thickening", "disease_findings": "Bronchial thickening in both lungs, more evident centrally", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_3709/train_3709_a/train_3709_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3709_a_2.nii.gz", "disease_mask": "seg_rxg/train_3709_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass opacities in the right upper lobe anterior segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_3709/train_3709_a/train_3709_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3709_a_2.nii.gz", "disease_mask": "seg_rxg/train_3709_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass opacities in the right lower lobe mediobasal segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_3709/train_3709_a/train_3709_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3709_a_2.nii.gz", "disease_mask": "seg_rxg/train_3709_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass opacities in the right lower lobe laterobasal segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_3709/train_3709_a/train_3709_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3709_a_2.nii.gz", "disease_mask": "seg_rxg/train_3709_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass opacities in the left lower lobe basal segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_3709/train_3709_a/train_3709_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3709_a_2.nii.gz", "disease_mask": "seg_rxg/train_3709_a_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass opacities in the left lower lobe anterobasal segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_3709/train_3709_a/train_3709_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3709_a_2.nii.gz", "disease_mask": "seg_rxg/train_3709_a_2.nii.gz", "disease_mask_channel": "7", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectatic changes in the right lower lobe laterobasal segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2861/train_2861_a/train_2861_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2861_a_2.nii.gz", "disease_mask": "seg_rxg/train_2861_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Minimal subpleural linear fibrotic changes in the lung parenchyma bilaterally", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2834/train_2834_a/train_2834_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2834_a_2.nii.gz", "disease_mask": "seg_rxg/train_2834_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2986/train_2986_d/train_2986_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2986_d_2.nii.gz", "disease_mask": "seg_rxg/train_2986_d_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Significant centriacinar opacities in the upper lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_2986/train_2986_d/train_2986_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2986_d_2.nii.gz", "disease_mask": "seg_rxg/train_2986_d_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass nodules in the upper lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2986/train_2986_d/train_2986_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2986_d_2.nii.gz", "disease_mask": "seg_rxg/train_2986_d_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Bronchial wall thickening", "disease_findings": "Bilateral peribronchial thickenings", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_2886/train_2886_a/train_2886_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2886_a_2.nii.gz", "disease_mask": "seg_rxg/train_2886_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Minimal free pleural effusion measuring 6 mm in thickness on the left", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_2886/train_2886_a/train_2886_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2886_a_2.nii.gz", "disease_mask": "seg_rxg/train_2886_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Branching opacities with buds and acinar infiltrates in the superior lingular segment of the left upper lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2886/train_2886_a/train_2886_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2886_a_2.nii.gz", "disease_mask": "seg_rxg/train_2886_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Newly emerging focal consolidation areas in several foci in different locations within the apicoposterior segment of the left upper lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2886/train_2886_a/train_2886_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2886_a_2.nii.gz", "disease_mask": "seg_rxg/train_2886_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Newly emerging nodular consolidation areas measuring 1 cm in diameter in the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2886/train_2886_a/train_2886_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2886_a_2.nii.gz", "disease_mask": "seg_rxg/train_2886_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Focal consolidation area in the laterobasal segment of the right lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2886/train_2886_a/train_2886_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2886_a_2.nii.gz", "disease_mask": "seg_rxg/train_2886_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Bronchial wall thickening", "disease_findings": "Bilateral peribronchial thickening", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_2514/train_2514_a/train_2514_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2514_a_2.nii.gz", "disease_mask": "seg_rxg/train_2514_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific nodules measuring approximately 7.5 mm in diameter, the largest in the right middle lobe, in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_3695/train_3695_a/train_3695_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3695_a_2.nii.gz", "disease_mask": "seg_rxg/train_3695_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Nodular ground glass densities in the posterobasal segments of the lower lobes in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2507/train_2507_a/train_2507_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2507_a_2.nii.gz", "disease_mask": "seg_rxg/train_2507_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Multiple round-shaped consolidations in both lungs, predominantly in the peripheral regions of the lower lobes", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2507/train_2507_a/train_2507_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2507_a_2.nii.gz", "disease_mask": "seg_rxg/train_2507_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in both lungs, predominantly in the peripheral regions of the lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_a/train_2718_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_a_2.nii.gz", "disease_mask": "seg_rxg/train_2718_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Bilateral pleural effusions", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_a/train_2718_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_a_2.nii.gz", "disease_mask": "seg_rxg/train_2718_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Centriacinar emphysema in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_a/train_2718_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_a_2.nii.gz", "disease_mask": "seg_rxg/train_2718_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Emphysema", "disease_findings": "Paraseptal emphysema in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_a/train_2718_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_a_2.nii.gz", "disease_mask": "seg_rxg/train_2718_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Pleural scarring densities in both lung apices", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_a/train_2718_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_a_2.nii.gz", "disease_mask": "seg_rxg/train_2718_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Parenchymal scarring densities in both lung apices", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_a/train_2718_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_a_2.nii.gz", "disease_mask": "seg_rxg/train_2718_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Linear", "disease_findings": "Pleural scarring densities in right lung middle lobe and upper lobe anterior segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_a/train_2718_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_a_2.nii.gz", "disease_mask": "seg_rxg/train_2718_a_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Linear", "disease_findings": "Parenchymal scarring densities in right lung middle lobe and upper lobe anterior segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_a/train_2718_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_a_2.nii.gz", "disease_mask": "seg_rxg/train_2718_a_2.nii.gz", "disease_mask_channel": "7", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subpleural nodule measuring 4 mm in diameter in middle lobe of right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2718/train_2718_a/train_2718_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2718_a_2.nii.gz", "disease_mask": "seg_rxg/train_2718_a_2.nii.gz", "disease_mask_channel": "8", "disease_label": "Septal thickening", "disease_findings": "Interlobular septal thickening in both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_2940/train_2940_a/train_2940_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2940_a_2.nii.gz", "disease_mask": "seg_rxg/train_2940_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass densities in the mediobasal segment of the lower lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5460/train_5460_b/train_5460_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5460_b_2.nii.gz", "disease_mask": "seg_rxg/train_5460_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral and centrally located ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5460/train_5460_b/train_5460_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5460_b_2.nii.gz", "disease_mask": "seg_rxg/train_5460_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear densities accompanying ground glass opacities", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5460/train_5460_b/train_5460_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5460_b_2.nii.gz", "disease_mask": "seg_rxg/train_5460_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Occasional consolidations accompanying ground glass opacities", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5254/train_5254_a/train_5254_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5254_a_2.nii.gz", "disease_mask": "seg_rxg/train_5254_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Reticulated nodular fibrotic density increases in the apices of both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_6377/train_6377_d/train_6377_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6377_d_2.nii.gz", "disease_mask": "seg_rxg/train_6377_d_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_6377/train_6377_d/train_6377_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6377_d_2.nii.gz", "disease_mask": "seg_rxg/train_6377_d_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Scarring in both lung apices", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6377/train_6377_d/train_6377_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6377_d_2.nii.gz", "disease_mask": "seg_rxg/train_6377_d_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectatic changes in both lungs, more prominent centrally", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_8150/train_8150_a/train_8150_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8150_a_2.nii.gz", "disease_mask": "seg_rxg/train_8150_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Subpleural subcentimeter air cysts in the anterior upper lobe of the right lung", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_5740/train_5740_a/train_5740_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5740_a_2.nii.gz", "disease_mask": "seg_rxg/train_5740_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "5 mm nodule in the subpleural area near the 5th vertebral body in the superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7189/train_7189_b/train_7189_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7189_b_2.nii.gz", "disease_mask": "seg_rxg/train_7189_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Parenchymal changes consistent with scarring in both lungs, more prominent on the right", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7189/train_7189_b/train_7189_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7189_b_2.nii.gz", "disease_mask": "seg_rxg/train_7189_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Stable lesion in the right upper lobe apical segment measuring approximately 13x10 mm, consistent with fibrotic nodular formation and associated volume loss", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6373/train_6373_a/train_6373_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6373_a_2.nii.gz", "disease_mask": "seg_rxg/train_6373_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_6373/train_6373_a/train_6373_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6373_a_2.nii.gz", "disease_mask": "seg_rxg/train_6373_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Occasional linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7552/train_7552_c/train_7552_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7552_c_2.nii.gz", "disease_mask": "seg_rxg/train_7552_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Bilateral pleural effusions measuring up to 18 mm on the right and 25 mm on the left at their deepest points", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_7552/train_7552_c/train_7552_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7552_c_2.nii.gz", "disease_mask": "seg_rxg/train_7552_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Wider band atelectatic changes in the basal segments of both lower lobes", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7552/train_7552_c/train_7552_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7552_c_2.nii.gz", "disease_mask": "seg_rxg/train_7552_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Wider band atelectatic changes in areas adjacent to the effusion on the left", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7552/train_7552_c/train_7552_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7552_c_2.nii.gz", "disease_mask": "seg_rxg/train_7552_c_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Focal consolidation previously seen in the anterior segment of the left upper lobe has markedly reduced", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5588/train_5588_a/train_5588_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5588_a_2.nii.gz", "disease_mask": "seg_rxg/train_5588_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Calcified pulmonary nodules in both lungs consistent with sequelae", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6418/train_6418_a/train_6418_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6418_a_2.nii.gz", "disease_mask": "seg_rxg/train_6418_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6816/train_6816_a/train_6816_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6816_a_2.nii.gz", "disease_mask": "seg_rxg/train_6816_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Minimal peribronchial thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_6284/train_6284_a/train_6284_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6284_a_2.nii.gz", "disease_mask": "seg_rxg/train_6284_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Fibrotic changes in the bilateral lung apices", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7127/train_7127_a/train_7127_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7127_a_2.nii.gz", "disease_mask": "seg_rxg/train_7127_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripherally located patchy ground-glass densities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5634/train_5634_c/train_5634_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5634_c_2.nii.gz", "disease_mask": "seg_rxg/train_5634_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Solid nodule measuring 6.7x4 mm in the middle lobe lateral segment of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5634/train_5634_c/train_5634_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5634_c_2.nii.gz", "disease_mask": "seg_rxg/train_5634_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Solid nodule up to 6 mm in the anterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5634/train_5634_c/train_5634_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5634_c_2.nii.gz", "disease_mask": "seg_rxg/train_5634_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Two ground-glass nodules, the largest measuring 6 mm, in the anterior segment of the right upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5634/train_5634_c/train_5634_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5634_c_2.nii.gz", "disease_mask": "seg_rxg/train_5634_c_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Solid and ground-glass nodules in both lungs, which were not seen in the previous CT examination", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6345/train_6345_a/train_6345_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6345_a_2.nii.gz", "disease_mask": "seg_rxg/train_6345_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_8061/train_8061_a/train_8061_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8061_a_2.nii.gz", "disease_mask": "seg_rxg/train_8061_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Bilateral peribronchial thickenings", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_8061/train_8061_a/train_8061_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8061_a_2.nii.gz", "disease_mask": "seg_rxg/train_8061_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectasis areas in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_8061/train_8061_a/train_8061_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8061_a_2.nii.gz", "disease_mask": "seg_rxg/train_8061_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Calcified nonspecific pulmonary nodule measuring 5 mm in the subpleural region of the lower lobe of the left lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7531/train_7531_a/train_7531_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7531_a_2.nii.gz", "disease_mask": "seg_rxg/train_7531_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular consolidations with irregular borders in all lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7531/train_7531_a/train_7531_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7531_a_2.nii.gz", "disease_mask": "seg_rxg/train_7531_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities in all lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7531/train_7531_a/train_7531_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7531_a_2.nii.gz", "disease_mask": "seg_rxg/train_7531_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Infiltrates in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6497/train_6497_a/train_6497_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6497_a_2.nii.gz", "disease_mask": "seg_rxg/train_6497_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "3 mm nodule located peripherally in the superior segment of the anterior lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6497/train_6497_a/train_6497_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6497_a_2.nii.gz", "disease_mask": "seg_rxg/train_6497_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Patchy, subpleural acinar density increases in the dependent sections of the left lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7568/train_7568_a/train_7568_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7568_a_2.nii.gz", "disease_mask": "seg_rxg/train_7568_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_8052/train_8052_a/train_8052_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8052_a_2.nii.gz", "disease_mask": "seg_rxg/train_8052_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_8052/train_8052_a/train_8052_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8052_a_2.nii.gz", "disease_mask": "seg_rxg/train_8052_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7693/train_7693_a/train_7693_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7693_a_2.nii.gz", "disease_mask": "seg_rxg/train_7693_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6975/train_6975_a/train_6975_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6975_a_2.nii.gz", "disease_mask": "seg_rxg/train_6975_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Centriacinar emphysema areas in the apex of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_6975/train_6975_a/train_6975_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6975_a_2.nii.gz", "disease_mask": "seg_rxg/train_6975_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Bleb formations in the anterior and apicoposterior segments of the left upper lobe and the mediobasal segment of the left lower lobe", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_6884/train_6884_a/train_6884_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6884_a_2.nii.gz", "disease_mask": "seg_rxg/train_6884_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectasis in a band-like pattern in the anterior lower lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6884/train_6884_a/train_6884_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6884_a_2.nii.gz", "disease_mask": "seg_rxg/train_6884_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in both lower lobes, more prominent on the left, with a tendency to merge with minimal borders", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6884/train_6884_a/train_6884_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6884_a_2.nii.gz", "disease_mask": "seg_rxg/train_6884_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter subpleural calcific nodule in the posterobasal region of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6521/train_6521_c/train_6521_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6521_c_2.nii.gz", "disease_mask": "seg_rxg/train_6521_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Minimal peribronchial thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_6521/train_6521_c/train_6521_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6521_c_2.nii.gz", "disease_mask": "seg_rxg/train_6521_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the lower lobe of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6521/train_6521_c/train_6521_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6521_c_2.nii.gz", "disease_mask": "seg_rxg/train_6521_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5799/train_5799_a/train_5799_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5799_a_2.nii.gz", "disease_mask": "seg_rxg/train_5799_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Slight atelectatic changes in the lung parenchyma adjacent to the osteophytes, more prominent on the right", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_8006/train_8006_a/train_8006_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8006_a_2.nii.gz", "disease_mask": "seg_rxg/train_8006_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Peripheral parenchymal nodules in both lungs, largest measuring 3.5 mm in diameter in the lateral basal segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_8006/train_8006_a/train_8006_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8006_a_2.nii.gz", "disease_mask": "seg_rxg/train_8006_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Increased reticular density of thymus residue in the anterior mediastinum", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_5988/train_5988_a/train_5988_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5988_a_2.nii.gz", "disease_mask": "seg_rxg/train_5988_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Fibrotic density in the medial segment of the right middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_8142/train_8142_a/train_8142_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8142_a_2.nii.gz", "disease_mask": "seg_rxg/train_8142_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_8142/train_8142_a/train_8142_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8142_a_2.nii.gz", "disease_mask": "seg_rxg/train_8142_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the right middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_8142/train_8142_a/train_8142_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8142_a_2.nii.gz", "disease_mask": "seg_rxg/train_8142_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the left upper lobe lingular segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_8142/train_8142_a/train_8142_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8142_a_2.nii.gz", "disease_mask": "seg_rxg/train_8142_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the left lower lobe basal segments", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_8142/train_8142_a/train_8142_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8142_a_2.nii.gz", "disease_mask": "seg_rxg/train_8142_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5212/train_5212_a/train_5212_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5212_a_2.nii.gz", "disease_mask": "seg_rxg/train_5212_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "9 mm calcified nodule on the diaphragmatic side of the lower lobe of the left lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_h/train_7250_h_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_h_2.nii.gz", "disease_mask": "seg_rxg/train_7250_h_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodule in the superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_h/train_7250_h_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_h_2.nii.gz", "disease_mask": "seg_rxg/train_7250_h_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_h/train_7250_h_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_h_2.nii.gz", "disease_mask": "seg_rxg/train_7250_h_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Occasional atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6156/train_6156_a/train_6156_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6156_a_2.nii.gz", "disease_mask": "seg_rxg/train_6156_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "2 mm pleural nodule in the apicoposterior segment of the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7944/train_7944_a/train_7944_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7944_a_2.nii.gz", "disease_mask": "seg_rxg/train_7944_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectasis in the central parts of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_7944/train_7944_a/train_7944_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7944_a_2.nii.gz", "disease_mask": "seg_rxg/train_7944_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Scarring in both lung apices", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7944/train_7944_a/train_7944_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7944_a_2.nii.gz", "disease_mask": "seg_rxg/train_7944_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the medial segment of the right middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7944/train_7944_a/train_7944_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7944_a_2.nii.gz", "disease_mask": "seg_rxg/train_7944_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the lingular segment of the left upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7944/train_7944_a/train_7944_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7944_a_2.nii.gz", "disease_mask": "seg_rxg/train_7944_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6779/train_6779_f/train_6779_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6779_f_2.nii.gz", "disease_mask": "seg_rxg/train_6779_f_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Bilateral pleural effusions measuring 26 mm on the right and 29 mm on the left", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_6779/train_6779_f/train_6779_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6779_f_2.nii.gz", "disease_mask": "seg_rxg/train_6779_f_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis associated with pleural effusions, especially on the left", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6779/train_6779_f/train_6779_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6779_f_2.nii.gz", "disease_mask": "seg_rxg/train_6779_f_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectatic changes in the right middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5915/train_5915_c/train_5915_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5915_c_2.nii.gz", "disease_mask": "seg_rxg/train_5915_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Moderate pleural effusion on the right", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5915/train_5915_c/train_5915_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5915_c_2.nii.gz", "disease_mask": "seg_rxg/train_5915_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Small to moderate pleural effusion on the left", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5915/train_5915_c/train_5915_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5915_c_2.nii.gz", "disease_mask": "seg_rxg/train_5915_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Septal thickening", "disease_findings": "Interlobular septal thickenings in both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_5915/train_5915_c/train_5915_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5915_c_2.nii.gz", "disease_mask": "seg_rxg/train_5915_c_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5915/train_5915_c/train_5915_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5915_c_2.nii.gz", "disease_mask": "seg_rxg/train_5915_c_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectatic changes associated with pleural effusion", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6196/train_6196_a/train_6196_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6196_a_2.nii.gz", "disease_mask": "seg_rxg/train_6196_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Fibroatelectasis causing volume loss and structural distortion in the medial segment of the right middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6196/train_6196_a/train_6196_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6196_a_2.nii.gz", "disease_mask": "seg_rxg/train_6196_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific subcentimeter calcific nodules in the medial segment of the right middle lobe, largest measuring 3.8 mm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5597/train_5597_a/train_5597_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5597_a_2.nii.gz", "disease_mask": "seg_rxg/train_5597_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectasis in the central parts of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_7978/train_7978_a/train_7978_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7978_a_2.nii.gz", "disease_mask": "seg_rxg/train_7978_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion on the right, measuring 3.0 cm at its thickest point", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_7978/train_7978_a/train_7978_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7978_a_2.nii.gz", "disease_mask": "seg_rxg/train_7978_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Minimal pleural effusion on the left", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_7978/train_7978_a/train_7978_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7978_a_2.nii.gz", "disease_mask": "seg_rxg/train_7978_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the lower lobes of both lungs adjacent to the pleural effusion", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7978/train_7978_a/train_7978_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7978_a_2.nii.gz", "disease_mask": "seg_rxg/train_7978_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the lower lobes of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7978/train_7978_a/train_7978_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7978_a_2.nii.gz", "disease_mask": "seg_rxg/train_7978_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the upper lobe of the left lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7978/train_7978_a/train_7978_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7978_a_2.nii.gz", "disease_mask": "seg_rxg/train_7978_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the middle lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7978/train_7978_a/train_7978_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7978_a_2.nii.gz", "disease_mask": "seg_rxg/train_7978_a_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_7978/train_7978_a/train_7978_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7978_a_2.nii.gz", "disease_mask": "seg_rxg/train_7978_a_2.nii.gz", "disease_mask_channel": "7", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7007/train_7007_a/train_7007_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7007_a_2.nii.gz", "disease_mask": "seg_rxg/train_7007_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest nodule measuring 4.5 mm in the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5751/train_5751_b/train_5751_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5751_b_2.nii.gz", "disease_mask": "seg_rxg/train_5751_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Mild scarring at the apical level of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6566/train_6566_a/train_6566_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6566_a_2.nii.gz", "disease_mask": "seg_rxg/train_6566_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Bilateral peribronchial thickening", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_7355/train_7355_a/train_7355_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7355_a_2.nii.gz", "disease_mask": "seg_rxg/train_7355_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Peripheral and central consolidations in the upper and lower lobes of both lungs and the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7355/train_7355_a/train_7355_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7355_a_2.nii.gz", "disease_mask": "seg_rxg/train_7355_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities in the upper and lower lobes of both lungs and the middle lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7355/train_7355_a/train_7355_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7355_a_2.nii.gz", "disease_mask": "seg_rxg/train_7355_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Septal thickening", "disease_findings": "Interlobular septal thickening within the ground glass areas", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_7355/train_7355_a/train_7355_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7355_a_2.nii.gz", "disease_mask": "seg_rxg/train_7355_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Linear atelectasis occasionally seen in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7355/train_7355_a/train_7355_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7355_a_2.nii.gz", "disease_mask": "seg_rxg/train_7355_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Findings consistent with viral pneumonia in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5774/train_5774_a/train_5774_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5774_a_2.nii.gz", "disease_mask": "seg_rxg/train_5774_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Fibrotic changes in the upper lobes of both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_5812/train_5812_a/train_5812_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5812_a_2.nii.gz", "disease_mask": "seg_rxg/train_5812_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral subpleural ground-glass opacities in the posterobasal segment of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7348/train_7348_a/train_7348_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7348_a_2.nii.gz", "disease_mask": "seg_rxg/train_7348_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subpleural subcentimeter calcific nodule in the laterobasal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_f/train_7250_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_f_2.nii.gz", "disease_mask": "seg_rxg/train_7250_f_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectasis in the central parts of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_f/train_7250_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_f_2.nii.gz", "disease_mask": "seg_rxg/train_7250_f_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule measuring 22 mm in the peripheral area of the medial superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_f/train_7250_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_f_2.nii.gz", "disease_mask": "seg_rxg/train_7250_f_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodule in the superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_f/train_7250_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_f_2.nii.gz", "disease_mask": "seg_rxg/train_7250_f_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_f/train_7250_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_f_2.nii.gz", "disease_mask": "seg_rxg/train_7250_f_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_8201/train_8201_a/train_8201_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8201_a_2.nii.gz", "disease_mask": "seg_rxg/train_8201_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Areas of linear subsegmental atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_8201/train_8201_a/train_8201_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8201_a_2.nii.gz", "disease_mask": "seg_rxg/train_8201_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "3 mm semisolid nodular lesion in the anterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5281/train_5281_a/train_5281_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5281_a_2.nii.gz", "disease_mask": "seg_rxg/train_5281_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Slightly irregularly circumscribed nodule measuring 8 mm in diameter in the posterobasal segment of the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5281/train_5281_a/train_5281_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5281_a_2.nii.gz", "disease_mask": "seg_rxg/train_5281_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Minimal volume loss around the nodule in the posterobasal segment of the lower lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5281/train_5281_a/train_5281_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5281_a_2.nii.gz", "disease_mask": "seg_rxg/train_5281_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear density increases around the nodule in the posterobasal segment of the lower lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5281/train_5281_a/train_5281_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5281_a_2.nii.gz", "disease_mask": "seg_rxg/train_5281_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Additional nodules in the right lung, the largest measuring approximately 5.3 mm in diameter in the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5281/train_5281_a/train_5281_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5281_a_2.nii.gz", "disease_mask": "seg_rxg/train_5281_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules in both lungs, more prominent on the right", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5529/train_5529_b/train_5529_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5529_b_2.nii.gz", "disease_mask": "seg_rxg/train_5529_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Residual soft tissue densities from primary mass around lower lobe bronchi in right lung hilum", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5529/train_5529_b/train_5529_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5529_b_2.nii.gz", "disease_mask": "seg_rxg/train_5529_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Collapsed lower lobe of left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5529/train_5529_b/train_5529_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5529_b_2.nii.gz", "disease_mask": "seg_rxg/train_5529_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of consolidation with air bronchograms in left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5529/train_5529_b/train_5529_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5529_b_2.nii.gz", "disease_mask": "seg_rxg/train_5529_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pleural effusion or thickening", "disease_findings": "Stable pleural effusion in left lung", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5529/train_5529_b/train_5529_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5529_b_2.nii.gz", "disease_mask": "seg_rxg/train_5529_b_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Emphysema", "disease_findings": "Centracinar emphysema in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_5529/train_5529_b/train_5529_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5529_b_2.nii.gz", "disease_mask": "seg_rxg/train_5529_b_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass opacities in middle and lower lobes of right lung with minimal increase in density compared to previous exam", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5529/train_5529_b/train_5529_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5529_b_2.nii.gz", "disease_mask": "seg_rxg/train_5529_b_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Stable primary mass in left lung hilum", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6289/train_6289_a/train_6289_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6289_a_2.nii.gz", "disease_mask": "seg_rxg/train_6289_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground-glass opacities predominantly in the peripheral lung tissue in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6289/train_6289_a/train_6289_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6289_a_2.nii.gz", "disease_mask": "seg_rxg/train_6289_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Ground-glass opacities with central consolidation", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6289/train_6289_a/train_6289_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6289_a_2.nii.gz", "disease_mask": "seg_rxg/train_6289_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Halo sign around some of the ground-glass nodules", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6810/train_6810_a/train_6810_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6810_a_2.nii.gz", "disease_mask": "seg_rxg/train_6810_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6810/train_6810_a/train_6810_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6810_a_2.nii.gz", "disease_mask": "seg_rxg/train_6810_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Calcified nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6810/train_6810_a/train_6810_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6810_a_2.nii.gz", "disease_mask": "seg_rxg/train_6810_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchial wall thickening", "disease_findings": "Diffuse peribronchial thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_6810/train_6810_a/train_6810_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6810_a_2.nii.gz", "disease_mask": "seg_rxg/train_6810_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Mosaic attenuation pattern in the lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6810/train_6810_a/train_6810_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6810_a_2.nii.gz", "disease_mask": "seg_rxg/train_6810_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Increased density consistent with linear atelectasis in the left upper lobe inferior lingular segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6810/train_6810_a/train_6810_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6810_a_2.nii.gz", "disease_mask": "seg_rxg/train_6810_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Linear", "disease_findings": "Increased density consistent with linear atelectasis in the right middle lobe medial segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7280/train_7280_c/train_7280_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7280_c_2.nii.gz", "disease_mask": "seg_rxg/train_7280_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Bilateral pleural effusion", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_7280/train_7280_c/train_7280_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7280_c_2.nii.gz", "disease_mask": "seg_rxg/train_7280_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion extending to the right lung apex in the supine position", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_7280/train_7280_c/train_7280_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7280_c_2.nii.gz", "disease_mask": "seg_rxg/train_7280_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis adjacent to the effusion in the lower lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7280/train_7280_c/train_7280_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7280_c_2.nii.gz", "disease_mask": "seg_rxg/train_7280_c_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple parenchymal nodules with irregular borders in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7280/train_7280_c/train_7280_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7280_c_2.nii.gz", "disease_mask": "seg_rxg/train_7280_c_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass area with interlobular septal thickening in the posterior segment of the right upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7280/train_7280_c/train_7280_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7280_c_2.nii.gz", "disease_mask": "seg_rxg/train_7280_c_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Groundglass opacity", "disease_findings": "Widespread and randomized distribution of ground glass area with interlobular septal thickening in the posterior segment of the right upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7507/train_7507_b/train_7507_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7507_b_2.nii.gz", "disease_mask": "seg_rxg/train_7507_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation area containing air bronchograms in the posterobasal part of the lower lobe of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7507/train_7507_b/train_7507_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7507_b_2.nii.gz", "disease_mask": "seg_rxg/train_7507_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass density in the posterobasal part of the lower lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7507/train_7507_b/train_7507_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7507_b_2.nii.gz", "disease_mask": "seg_rxg/train_7507_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Findings suggestive of pneumonic infiltration in both lungs, more prominently in the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5536/train_5536_b/train_5536_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5536_b_2.nii.gz", "disease_mask": "seg_rxg/train_5536_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Increase in the distribution of pneumonic infiltrates in the lung parenchyma", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_8106/train_8106_a/train_8106_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8106_a_2.nii.gz", "disease_mask": "seg_rxg/train_8106_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Multilobar, multisegmentary central-peripherally located nodular-patchy ground glass opacities forming a crazy paving pattern in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_8106/train_8106_a/train_8106_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8106_a_2.nii.gz", "disease_mask": "seg_rxg/train_8106_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the right middle lobe, left lower lingular segment, and basal segments of both lower lobes", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_8106/train_8106_a/train_8106_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8106_a_2.nii.gz", "disease_mask": "seg_rxg/train_8106_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Subpleural lines in the right middle lobe, left lower lingular segment, and basal segments of both lower lobes", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7119/train_7119_a/train_7119_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7119_a_2.nii.gz", "disease_mask": "seg_rxg/train_7119_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "3 mm nodule in the lateral basal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7096/train_7096_a/train_7096_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7096_a_2.nii.gz", "disease_mask": "seg_rxg/train_7096_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Peripheral consolidation in the anteromediobasal segment of the left lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7491/train_7491_a/train_7491_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7491_a_2.nii.gz", "disease_mask": "seg_rxg/train_7491_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Diffuse interlobular septal thickening in the lower lobes of both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_7491/train_7491_a/train_7491_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7491_a_2.nii.gz", "disease_mask": "seg_rxg/train_7491_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Mild bilateral pleural effusion", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_7491/train_7491_a/train_7491_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7491_a_2.nii.gz", "disease_mask": "seg_rxg/train_7491_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7918/train_7918_a/train_7918_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7918_a_2.nii.gz", "disease_mask": "seg_rxg/train_7918_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Mild atelectatic changes in the paracardiac area anteriorly in the upper lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_8198/train_8198_b/train_8198_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8198_b_2.nii.gz", "disease_mask": "seg_rxg/train_8198_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of consolidation with indistinct margins in the lower lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_8198/train_8198_b/train_8198_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8198_b_2.nii.gz", "disease_mask": "seg_rxg/train_8198_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Increased ground-glass opacity in the lower lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_8198/train_8198_b/train_8198_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8198_b_2.nii.gz", "disease_mask": "seg_rxg/train_8198_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of consolidation with a tree-in-bud appearance in the upper lobe inferior lingular segment of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_8198/train_8198_b/train_8198_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8198_b_2.nii.gz", "disease_mask": "seg_rxg/train_8198_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of consolidation with a tree-in-bud appearance in the upper lobe anterior segment of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_8198/train_8198_b/train_8198_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8198_b_2.nii.gz", "disease_mask": "seg_rxg/train_8198_b_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pleural effusion or thickening", "disease_findings": "Minimal right pleural effusion", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5680/train_5680_a/train_5680_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5680_a_2.nii.gz", "disease_mask": "seg_rxg/train_5680_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Bilateral tree-in-bud opacities more prominent in the right middle lobe and the superior segment of the right upper lobe", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_5680/train_5680_a/train_5680_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5680_a_2.nii.gz", "disease_mask": "seg_rxg/train_5680_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules measuring up to 11 mm in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5680/train_5680_a/train_5680_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5680_a_2.nii.gz", "disease_mask": "seg_rxg/train_5680_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several larger nodules in the anterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6317/train_6317_a/train_6317_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6317_a_2.nii.gz", "disease_mask": "seg_rxg/train_6317_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral ground-glass opacities in the lower lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6317/train_6317_a/train_6317_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6317_a_2.nii.gz", "disease_mask": "seg_rxg/train_6317_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral ground-glass opacities in the middle lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6317/train_6317_a/train_6317_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6317_a_2.nii.gz", "disease_mask": "seg_rxg/train_6317_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Round ground-glass opacities", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6317/train_6317_a/train_6317_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6317_a_2.nii.gz", "disease_mask": "seg_rxg/train_6317_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities accompanied by interlobular septal thickening", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5803/train_5803_a/train_5803_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5803_a_2.nii.gz", "disease_mask": "seg_rxg/train_5803_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Areas of alveolar infiltrates with ground glass density in all lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5803/train_5803_a/train_5803_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5803_a_2.nii.gz", "disease_mask": "seg_rxg/train_5803_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Diffuse parenchymal confluence in all lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5803/train_5803_a/train_5803_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5803_a_2.nii.gz", "disease_mask": "seg_rxg/train_5803_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Extensive lung parenchymal involvement of COVID-19 infection", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7514/train_7514_a/train_7514_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7514_a_2.nii.gz", "disease_mask": "seg_rxg/train_7514_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the lingular segment of the upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5831/train_5831_a/train_5831_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5831_a_2.nii.gz", "disease_mask": "seg_rxg/train_5831_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Multilobar, multisegmental areas of ground glass opacity in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5831/train_5831_a/train_5831_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5831_a_2.nii.gz", "disease_mask": "seg_rxg/train_5831_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7294/train_7294_a/train_7294_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7294_a_2.nii.gz", "disease_mask": "seg_rxg/train_7294_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Central-peripheral localized nodular ground glass opacities with faint borders in the basal segment of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7294/train_7294_a/train_7294_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7294_a_2.nii.gz", "disease_mask": "seg_rxg/train_7294_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific calcific nodule in the posterior subsegment of the left upper lobe apicoposterior segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7597/train_7597_a/train_7597_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7597_a_2.nii.gz", "disease_mask": "seg_rxg/train_7597_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "2 mm diameter nodule in the anterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7597/train_7597_a/train_7597_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7597_a_2.nii.gz", "disease_mask": "seg_rxg/train_7597_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear areas of atelectasis in the upper lobe of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7597/train_7597_a/train_7597_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7597_a_2.nii.gz", "disease_mask": "seg_rxg/train_7597_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear areas of atelectasis in the lateral segment of the right middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7597/train_7597_a/train_7597_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7597_a_2.nii.gz", "disease_mask": "seg_rxg/train_7597_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Linear areas of atelectasis in the lingular segment of the left upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7597/train_7597_a/train_7597_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7597_a_2.nii.gz", "disease_mask": "seg_rxg/train_7597_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Linear areas of atelectasis in the medial segment of the left lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7597/train_7597_a/train_7597_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7597_a_2.nii.gz", "disease_mask": "seg_rxg/train_7597_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter parenchymal air cyst in the superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5699/train_5699_a/train_5699_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5699_a_2.nii.gz", "disease_mask": "seg_rxg/train_5699_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities predominantly peripherally located in the upper lobes on the left", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5699/train_5699_a/train_5699_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5699_a_2.nii.gz", "disease_mask": "seg_rxg/train_5699_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities more prominent in the lower lobes on the right", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5699/train_5699_a/train_5699_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5699_a_2.nii.gz", "disease_mask": "seg_rxg/train_5699_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules in the posterior upper lobe of the left lung with the largest measuring 4 mm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5699/train_5699_a/train_5699_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5699_a_2.nii.gz", "disease_mask": "seg_rxg/train_5699_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules at the level of the left lung lingular segment situated in the major fissures", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6346/train_6346_a/train_6346_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6346_a_2.nii.gz", "disease_mask": "seg_rxg/train_6346_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5922/train_5922_a/train_5922_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5922_a_2.nii.gz", "disease_mask": "seg_rxg/train_5922_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter subpleural nonspecific nodules in the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6541/train_6541_a/train_6541_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6541_a_2.nii.gz", "disease_mask": "seg_rxg/train_6541_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_6541/train_6541_a/train_6541_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6541_a_2.nii.gz", "disease_mask": "seg_rxg/train_6541_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Calcific nodules in the lung parenchyma, more prominent in the left lower lobe laterobasal segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5250/train_5250_a/train_5250_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5250_a_2.nii.gz", "disease_mask": "seg_rxg/train_5250_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Peripheral consolidations in the form of peribronchial patches in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5178/train_5178_a/train_5178_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5178_a_2.nii.gz", "disease_mask": "seg_rxg/train_5178_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_5178/train_5178_a/train_5178_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5178_a_2.nii.gz", "disease_mask": "seg_rxg/train_5178_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Subsegmental linear atelectasis in the right middle lobe lateral segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5178/train_5178_a/train_5178_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5178_a_2.nii.gz", "disease_mask": "seg_rxg/train_5178_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Subsegmental linear atelectasis in the left upper lobe inferior lingular segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5178/train_5178_a/train_5178_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5178_a_2.nii.gz", "disease_mask": "seg_rxg/train_5178_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Pleural-based, non-specific 5 mm pulmonary nodule in the laterobasal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5178/train_5178_a/train_5178_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5178_a_2.nii.gz", "disease_mask": "seg_rxg/train_5178_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter non-specific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5231/train_5231_a/train_5231_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5231_a_2.nii.gz", "disease_mask": "seg_rxg/train_5231_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Peripheral and central consolidations in the upper and lower lobes of both lungs and the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5231/train_5231_a/train_5231_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5231_a_2.nii.gz", "disease_mask": "seg_rxg/train_5231_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the upper and lower lobes of both lungs and the middle lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5231/train_5231_a/train_5231_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5231_a_2.nii.gz", "disease_mask": "seg_rxg/train_5231_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest lesions located in the posterobasal segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5947/train_5947_a/train_5947_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5947_a_2.nii.gz", "disease_mask": "seg_rxg/train_5947_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear passive atelectatic changes in the mediobasal segment of the right lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5947/train_5947_a/train_5947_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5947_a_2.nii.gz", "disease_mask": "seg_rxg/train_5947_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear passive atelectatic changes in the posterobasal segment of the left lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5947/train_5947_a/train_5947_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5947_a_2.nii.gz", "disease_mask": "seg_rxg/train_5947_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Increased pleuroparenchymal densities in both lung apical segments", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5222/train_5222_a/train_5222_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5222_a_2.nii.gz", "disease_mask": "seg_rxg/train_5222_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Bilateral asymmetrical peribronchial and subpleural ground-glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5222/train_5222_a/train_5222_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5222_a_2.nii.gz", "disease_mask": "seg_rxg/train_5222_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of consolidation in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5222/train_5222_a/train_5222_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5222_a_2.nii.gz", "disease_mask": "seg_rxg/train_5222_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Bilateral atypical pneumonic infiltration areas in the lung parenchyma", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5222/train_5222_a/train_5222_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5222_a_2.nii.gz", "disease_mask": "seg_rxg/train_5222_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Parenchymal involvement of COVID-19 infection", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7165/train_7165_a/train_7165_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7165_a_2.nii.gz", "disease_mask": "seg_rxg/train_7165_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Areas of linear-subsegmental atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7165/train_7165_a/train_7165_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7165_a_2.nii.gz", "disease_mask": "seg_rxg/train_7165_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several nodules with a diameter of 4 mm in both lungs, largest in the lateral segment of the lower lobe of the left lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7009/train_7009_a/train_7009_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7009_a_2.nii.gz", "disease_mask": "seg_rxg/train_7009_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter subpleural nodule at the posterobasal level of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7009/train_7009_a/train_7009_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7009_a_2.nii.gz", "disease_mask": "seg_rxg/train_7009_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter subpleural nodule in the lateral lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7009/train_7009_a/train_7009_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7009_a_2.nii.gz", "disease_mask": "seg_rxg/train_7009_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Mild linear atelectasis at the basal level of the left lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7758/train_7758_b/train_7758_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7758_b_2.nii.gz", "disease_mask": "seg_rxg/train_7758_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few subcentimeter nodules, some of which are calcified and nonspecific, in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6192/train_6192_b/train_6192_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6192_b_2.nii.gz", "disease_mask": "seg_rxg/train_6192_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Scattered and patchy ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6192/train_6192_b/train_6192_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6192_b_2.nii.gz", "disease_mask": "seg_rxg/train_6192_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "More prominent ground glass opacities in the left lung and subpleural area", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6192/train_6192_b/train_6192_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6192_b_2.nii.gz", "disease_mask": "seg_rxg/train_6192_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear area of atelectasis in the lower lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5691/train_5691_c/train_5691_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5691_c_2.nii.gz", "disease_mask": "seg_rxg/train_5691_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectasis in the central parts of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_5691/train_5691_c/train_5691_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5691_c_2.nii.gz", "disease_mask": "seg_rxg/train_5691_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Diffuse emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_5691/train_5691_c/train_5691_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5691_c_2.nii.gz", "disease_mask": "seg_rxg/train_5691_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several nonspecific nodules in both lungs measuring approximately 4 mm in diameter", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5691/train_5691_c/train_5691_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5691_c_2.nii.gz", "disease_mask": "seg_rxg/train_5691_c_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest nodule in the lateral part of the left lung upper lobe apicoposterior segment posterior subsegment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5691/train_5691_c/train_5691_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5691_c_2.nii.gz", "disease_mask": "seg_rxg/train_5691_c_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Occasional linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7406/train_7406_a/train_7406_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7406_a_2.nii.gz", "disease_mask": "seg_rxg/train_7406_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules up to 6 mm in the superior segment of the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7406/train_7406_a/train_7406_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7406_a_2.nii.gz", "disease_mask": "seg_rxg/train_7406_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass density nodules in the basal segments of both lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7406/train_7406_a/train_7406_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7406_a_2.nii.gz", "disease_mask": "seg_rxg/train_7406_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules measuring 4 mm in the anterior segment of the upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7406/train_7406_a/train_7406_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7406_a_2.nii.gz", "disease_mask": "seg_rxg/train_7406_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules measuring 4 mm in diameter in the posterior segment of the upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7406/train_7406_a/train_7406_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7406_a_2.nii.gz", "disease_mask": "seg_rxg/train_7406_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules measuring 3-4 mm in diameter in the middle lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7406/train_7406_a/train_7406_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7406_a_2.nii.gz", "disease_mask": "seg_rxg/train_7406_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules measuring 5.5 mm and 4 mm in diameter in the anterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7406/train_7406_a/train_7406_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7406_a_2.nii.gz", "disease_mask": "seg_rxg/train_7406_a_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Linear", "disease_findings": "Mild pleuroparenchymal density increases in the subpleural regions at the apex of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7189/train_7189_a/train_7189_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7189_a_2.nii.gz", "disease_mask": "seg_rxg/train_7189_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodular density in the apical segment of the upper lobe of the right lung, 13x5 mm in the peripheral area", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7189/train_7189_a/train_7189_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7189_a_2.nii.gz", "disease_mask": "seg_rxg/train_7189_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear density increases around the nodular density in the right lung apex", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7189/train_7189_a/train_7189_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7189_a_2.nii.gz", "disease_mask": "seg_rxg/train_7189_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Scarring in the right lung apex", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7189/train_7189_a/train_7189_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7189_a_2.nii.gz", "disease_mask": "seg_rxg/train_7189_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Linear atelectasis in both lungs, more prominently in the lower lobes", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7189/train_7189_a/train_7189_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7189_a_2.nii.gz", "disease_mask": "seg_rxg/train_7189_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_7189/train_7189_a/train_7189_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7189_a_2.nii.gz", "disease_mask": "seg_rxg/train_7189_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6305/train_6305_a/train_6305_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6305_a_2.nii.gz", "disease_mask": "seg_rxg/train_6305_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripherally located subpleural nodular ground glass densities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5963/train_5963_a/train_5963_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5963_a_2.nii.gz", "disease_mask": "seg_rxg/train_5963_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Calcified, well-circumscribed nodule measuring 5 mm in the lateral segment of the right middle lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5963/train_5963_a/train_5963_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5963_a_2.nii.gz", "disease_mask": "seg_rxg/train_5963_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Calcified nodules with irregular borders measuring 14 mm and 10 mm in the anterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5963/train_5963_a/train_5963_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5963_a_2.nii.gz", "disease_mask": "seg_rxg/train_5963_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Micronodules", "disease_findings": "Mosaic attenuation pattern in both lung parenchyma", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_5385/train_5385_a/train_5385_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5385_a_2.nii.gz", "disease_mask": "seg_rxg/train_5385_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass densities with a halo sign in the posterobasal segments of the lower lobes of both lungs in a subcentimeter patchwork pattern", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5483/train_5483_a/train_5483_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5483_a_2.nii.gz", "disease_mask": "seg_rxg/train_5483_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Occasional areas of atelectasis in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5483/train_5483_a/train_5483_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5483_a_2.nii.gz", "disease_mask": "seg_rxg/train_5483_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass appearance in the peripheral area of the anterior segment of the right upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5483/train_5483_a/train_5483_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5483_a_2.nii.gz", "disease_mask": "seg_rxg/train_5483_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Septal thickening", "disease_findings": "Interlobular septal thickening in the peripheral area of the anterior segment of the right upper lobe", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_6764/train_6764_b/train_6764_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6764_b_2.nii.gz", "disease_mask": "seg_rxg/train_6764_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Scarring in the apices of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Pneumonic consolidation in the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchial wall thickening", "disease_findings": "Bronchial wall thickening in the middle lobe of the right lung", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Air bronchograms in the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Focal consolidation in the right lung lower lobe laterobasal segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the right lung lower lobe laterobasal segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Bronchial wall thickening", "disease_findings": "Peribronchial wall thickening in the left lung lower lobe laterobasal segment", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "7", "disease_label": "Atelectasis, consolidation", "disease_findings": "Focal consolidation in the left lung lower lobe laterobasal segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "8", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the left lung lower lobe laterobasal segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "9", "disease_label": "Bronchial wall thickening", "disease_findings": "Peribronchial wall thickening in the left lung lower lobe anterobasal segment", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "10", "disease_label": "Atelectasis, consolidation", "disease_findings": "Focal consolidation in the left lung lower lobe anterobasal segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6401/train_6401_a/train_6401_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6401_a_2.nii.gz", "disease_mask": "seg_rxg/train_6401_a_2.nii.gz", "disease_mask_channel": "11", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the left lung lower lobe anterobasal segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6192/train_6192_a/train_6192_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6192_a_2.nii.gz", "disease_mask": "seg_rxg/train_6192_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the posterobasal region of the right lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5640/train_5640_d/train_5640_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5640_d_2.nii.gz", "disease_mask": "seg_rxg/train_5640_d_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Drainage catheter placed within the consolidation and effusion area containing air densities in the right hemithorax", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5640/train_5640_d/train_5640_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5640_d_2.nii.gz", "disease_mask": "seg_rxg/train_5640_d_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Nonspecific ground glass opacities in the right middle lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5640/train_5640_d/train_5640_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5640_d_2.nii.gz", "disease_mask": "seg_rxg/train_5640_d_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Emphysema", "disease_findings": "Widespread areas of emphysema around the subcutaneous fat and muscle planes on both the anterior chest wall and right posterior thoracic wall", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_5677/train_5677_a/train_5677_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5677_a_2.nii.gz", "disease_mask": "seg_rxg/train_5677_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5757/train_5757_a/train_5757_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5757_a_2.nii.gz", "disease_mask": "seg_rxg/train_5757_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "2 mm nodule in the anterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5757/train_5757_a/train_5757_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5757_a_2.nii.gz", "disease_mask": "seg_rxg/train_5757_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "2 mm nodule in the apicoposterior segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7935/train_7935_a/train_7935_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7935_a_2.nii.gz", "disease_mask": "seg_rxg/train_7935_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subpleural 4 mm fibrotic nodule in the laterobasal segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7571/train_7571_a/train_7571_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7571_a_2.nii.gz", "disease_mask": "seg_rxg/train_7571_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation area in the posterobasal segment of the right lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7571/train_7571_a/train_7571_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7571_a_2.nii.gz", "disease_mask": "seg_rxg/train_7571_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground glass densities in the posterobasal segment of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6310/train_6310_a/train_6310_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6310_a_2.nii.gz", "disease_mask": "seg_rxg/train_6310_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities in both lungs without bilateral contour formation", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6310/train_6310_a/train_6310_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6310_a_2.nii.gz", "disease_mask": "seg_rxg/train_6310_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities predominantly located in the peribronchial region", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7379/train_7379_b/train_7379_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7379_b_2.nii.gz", "disease_mask": "seg_rxg/train_7379_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Passive atelectatic changes in the medial segment of the right middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7379/train_7379_b/train_7379_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7379_b_2.nii.gz", "disease_mask": "seg_rxg/train_7379_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear subsegmental atelectatic changes in the posterobasal segment of the left lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5915/train_5915_a/train_5915_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5915_a_2.nii.gz", "disease_mask": "seg_rxg/train_5915_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Free effusion up to 8 cm in the left pleural space", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5915/train_5915_a/train_5915_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5915_a_2.nii.gz", "disease_mask": "seg_rxg/train_5915_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Free effusion up to 3 cm in the right pleural space", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5915/train_5915_a/train_5915_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5915_a_2.nii.gz", "disease_mask": "seg_rxg/train_5915_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation area adjacent to the effusion in the anteromedial segment of the left lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5472/train_5472_a/train_5472_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5472_a_2.nii.gz", "disease_mask": "seg_rxg/train_5472_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion measuring 2.7 cm at its thickest part in the left hemithorax", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5472/train_5472_a/train_5472_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5472_a_2.nii.gz", "disease_mask": "seg_rxg/train_5472_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion extending into the fissure on the left", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5472/train_5472_a/train_5472_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5472_a_2.nii.gz", "disease_mask": "seg_rxg/train_5472_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis adjacent to the pleural effusion in the lower lobe of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5472/train_5472_a/train_5472_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5472_a_2.nii.gz", "disease_mask": "seg_rxg/train_5472_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the lingular segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5472/train_5472_a/train_5472_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5472_a_2.nii.gz", "disease_mask": "seg_rxg/train_5472_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Septal thickening", "disease_findings": "Bilateral interlobular septal thickening suggesting mild cardiac edema", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_6330/train_6330_a/train_6330_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6330_a_2.nii.gz", "disease_mask": "seg_rxg/train_6330_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few subcentimeter nonspecific nodules in the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5668/train_5668_a/train_5668_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5668_a_2.nii.gz", "disease_mask": "seg_rxg/train_5668_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Patchy diffuse ARDS in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5668/train_5668_a/train_5668_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5668_a_2.nii.gz", "disease_mask": "seg_rxg/train_5668_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities in a crazy paving pattern in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5668/train_5668_a/train_5668_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5668_a_2.nii.gz", "disease_mask": "seg_rxg/train_5668_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pleural effusion or thickening", "disease_findings": "Small amount of bilateral pleural effusion", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5428/train_5428_c/train_5428_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5428_c_2.nii.gz", "disease_mask": "seg_rxg/train_5428_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectatic changes in the lower lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5428/train_5428_c/train_5428_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5428_c_2.nii.gz", "disease_mask": "seg_rxg/train_5428_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectatic changes in the inferior lingular segment of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5428/train_5428_c/train_5428_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5428_c_2.nii.gz", "disease_mask": "seg_rxg/train_5428_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectatic changes in the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5428/train_5428_c/train_5428_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5428_c_2.nii.gz", "disease_mask": "seg_rxg/train_5428_c_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Emphysema", "disease_findings": "Mild emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_5428/train_5428_c/train_5428_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5428_c_2.nii.gz", "disease_mask": "seg_rxg/train_5428_c_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subpleural nodule measuring 3.3 mm in diameter in the apicoposterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5511/train_5511_a/train_5511_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5511_a_2.nii.gz", "disease_mask": "seg_rxg/train_5511_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Bronchial wall thickening with collapsed segmental bronchi and intraluminal secretions in the lower lobe basal segments", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_5511/train_5511_a/train_5511_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5511_a_2.nii.gz", "disease_mask": "seg_rxg/train_5511_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectasis in the inferior segment of the left lung lingula", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5511/train_5511_a/train_5511_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5511_a_2.nii.gz", "disease_mask": "seg_rxg/train_5511_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Bronchopneumonic infiltration in the anterior segment of the right upper lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6610/train_6610_a/train_6610_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6610_a_2.nii.gz", "disease_mask": "seg_rxg/train_6610_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few subcentimeter non-specific nodules", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5428/train_5428_i/train_5428_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5428_i_2.nii.gz", "disease_mask": "seg_rxg/train_5428_i_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Areas of increased density consistent with pleuroparenchymal scarring in the apical posterobasal section of the right upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5428/train_5428_i/train_5428_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5428_i_2.nii.gz", "disease_mask": "seg_rxg/train_5428_i_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Minimal regression in peripherally located ground-glass opacities in the left upper lobe compared to previous examination", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5428/train_5428_i/train_5428_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5428_i_2.nii.gz", "disease_mask": "seg_rxg/train_5428_i_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Septal thickening", "disease_findings": "Interseptal thickening in the left upper lobe", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_5428/train_5428_i/train_5428_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5428_i_2.nii.gz", "disease_mask": "seg_rxg/train_5428_i_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Increased pleuroparenchymal density in the posterobasal region of the right lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7346/train_7346_b/train_7346_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7346_b_2.nii.gz", "disease_mask": "seg_rxg/train_7346_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear fibroatelectasis changes in the left upper lobe lingular segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7346/train_7346_b/train_7346_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7346_b_2.nii.gz", "disease_mask": "seg_rxg/train_7346_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear fibroatelectasis changes in the left lower lobe anteromediobasal segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7902/train_7902_a/train_7902_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7902_a_2.nii.gz", "disease_mask": "seg_rxg/train_7902_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific parenchymal nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5460/train_5460_a/train_5460_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5460_a_2.nii.gz", "disease_mask": "seg_rxg/train_5460_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the right middle lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5460/train_5460_a/train_5460_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5460_a_2.nii.gz", "disease_mask": "seg_rxg/train_5460_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the posterobasal segments of the bilateral lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5460/train_5460_a/train_5460_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5460_a_2.nii.gz", "disease_mask": "seg_rxg/train_5460_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the peripheral subpleural areas", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5460/train_5460_a/train_5460_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5460_a_2.nii.gz", "disease_mask": "seg_rxg/train_5460_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter ground-glass nodule in the posterobasal segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5460/train_5460_a/train_5460_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5460_a_2.nii.gz", "disease_mask": "seg_rxg/train_5460_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis changes in the posterobasal segments of both lower lobes", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6368/train_6368_a/train_6368_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6368_a_2.nii.gz", "disease_mask": "seg_rxg/train_6368_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Tubular bronchiectatic changes with significant bronchial wall thickening in the upper lobe of the right lung", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_6368/train_6368_a/train_6368_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6368_a_2.nii.gz", "disease_mask": "seg_rxg/train_6368_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Associated volume loss in the upper lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6368/train_6368_a/train_6368_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6368_a_2.nii.gz", "disease_mask": "seg_rxg/train_6368_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Calcified nonspecific pleural nodules measuring 12x8 mm in the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6368/train_6368_a/train_6368_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6368_a_2.nii.gz", "disease_mask": "seg_rxg/train_6368_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Calcified nonspecific pleural nodules in the right middle lobe lateral segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6368/train_6368_a/train_6368_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6368_a_2.nii.gz", "disease_mask": "seg_rxg/train_6368_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest calcified nonspecific pleural nodule in the anterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7706/train_7706_c/train_7706_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7706_c_2.nii.gz", "disease_mask": "seg_rxg/train_7706_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation areas in the upper and lower lobes of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7706/train_7706_c/train_7706_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7706_c_2.nii.gz", "disease_mask": "seg_rxg/train_7706_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation area in the lower lobe of the right lung accompanied by atelectasis", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7706/train_7706_c/train_7706_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7706_c_2.nii.gz", "disease_mask": "seg_rxg/train_7706_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Emphysema", "disease_findings": "Air trapping areas in the posterior segments of the right upper lobe and left upper lobe", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_5680/train_5680_b/train_5680_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5680_b_2.nii.gz", "disease_mask": "seg_rxg/train_5680_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5680/train_5680_b/train_5680_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5680_b_2.nii.gz", "disease_mask": "seg_rxg/train_5680_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidations in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5680/train_5680_b/train_5680_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5680_b_2.nii.gz", "disease_mask": "seg_rxg/train_5680_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Predominantly peripheral ground-glass opacities and consolidations in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5680/train_5680_b/train_5680_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5680_b_2.nii.gz", "disease_mask": "seg_rxg/train_5680_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Band-like density increases in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7505/train_7505_b/train_7505_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7505_b_2.nii.gz", "disease_mask": "seg_rxg/train_7505_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Increase in prevalence and intensity of pneumonic infiltration in lung parenchyma", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7611/train_7611_a/train_7611_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7611_a_2.nii.gz", "disease_mask": "seg_rxg/train_7611_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities with multisegmental involvement, predominantly peripheral, in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_8164/train_8164_a/train_8164_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8164_a_2.nii.gz", "disease_mask": "seg_rxg/train_8164_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Parenchymal nodular consolidations in both lungs, predominantly in the posterior subpleural regions of the lower lobes", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_8164/train_8164_a/train_8164_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_8164_a_2.nii.gz", "disease_mask": "seg_rxg/train_8164_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in both lungs, predominantly in the posterior subpleural regions of the lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7062/train_7062_a/train_7062_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7062_a_2.nii.gz", "disease_mask": "seg_rxg/train_7062_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Patchy nodular densities in the posterior segment of the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7062/train_7062_a/train_7062_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7062_a_2.nii.gz", "disease_mask": "seg_rxg/train_7062_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Patchy nodular densities in the lateral and medial segments of the right middle lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7062/train_7062_a/train_7062_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7062_a_2.nii.gz", "disease_mask": "seg_rxg/train_7062_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Patchy nodular densities in all segments of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7062/train_7062_a/train_7062_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7062_a_2.nii.gz", "disease_mask": "seg_rxg/train_7062_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Micronodules", "disease_findings": "Tree-in-bud appearance of densities in the right lung", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_7062/train_7062_a/train_7062_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7062_a_2.nii.gz", "disease_mask": "seg_rxg/train_7062_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Patchy nodular densities in the lower lobes of the left lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7062/train_7062_a/train_7062_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7062_a_2.nii.gz", "disease_mask": "seg_rxg/train_7062_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Micronodules", "disease_findings": "Tree-in-bud appearance of densities in the lower lobes of the left lung", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_5999/train_5999_a/train_5999_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5999_a_2.nii.gz", "disease_mask": "seg_rxg/train_5999_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Scarring changes in the apex of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5999/train_5999_a/train_5999_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5999_a_2.nii.gz", "disease_mask": "seg_rxg/train_5999_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5999/train_5999_a/train_5999_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5999_a_2.nii.gz", "disease_mask": "seg_rxg/train_5999_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in the upper lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_6967/train_6967_a/train_6967_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6967_a_2.nii.gz", "disease_mask": "seg_rxg/train_6967_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7498/train_7498_a/train_7498_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7498_a_2.nii.gz", "disease_mask": "seg_rxg/train_7498_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Interlobular septal thickenings in the lower lobes of both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_7498/train_7498_a/train_7498_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7498_a_2.nii.gz", "disease_mask": "seg_rxg/train_7498_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the lower lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7498/train_7498_a/train_7498_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7498_a_2.nii.gz", "disease_mask": "seg_rxg/train_7498_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule measuring 6.6 mm in the anterior segment of the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7498/train_7498_a/train_7498_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7498_a_2.nii.gz", "disease_mask": "seg_rxg/train_7498_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule measuring 5 mm in the right middle lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7498/train_7498_a/train_7498_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7498_a_2.nii.gz", "disease_mask": "seg_rxg/train_7498_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule measuring 4.5 mm in the anterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7498/train_7498_a/train_7498_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7498_a_2.nii.gz", "disease_mask": "seg_rxg/train_7498_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule measuring 4 mm in the lingular segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7402/train_7402_a/train_7402_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7402_a_2.nii.gz", "disease_mask": "seg_rxg/train_7402_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Peribronchial thickening in both lower lobes of the lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_7402/train_7402_a/train_7402_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7402_a_2.nii.gz", "disease_mask": "seg_rxg/train_7402_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchial wall thickening", "disease_findings": "Tracheomalacia in both main bronchi and lobar bronchial walls", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_7402/train_7402_a/train_7402_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7402_a_2.nii.gz", "disease_mask": "seg_rxg/train_7402_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pleural effusion or thickening", "disease_findings": "Minimal pleural effusion on the right", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5491/train_5491_a/train_5491_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5491_a_2.nii.gz", "disease_mask": "seg_rxg/train_5491_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5307/train_5307_a/train_5307_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5307_a_2.nii.gz", "disease_mask": "seg_rxg/train_5307_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Minimal scarring changes in both lung apices", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6641/train_6641_a/train_6641_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6641_a_2.nii.gz", "disease_mask": "seg_rxg/train_6641_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules up to 5 mm in size, especially in the middle lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6641/train_6641_a/train_6641_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6641_a_2.nii.gz", "disease_mask": "seg_rxg/train_6641_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Paracardiac fibrotic changes in the medial right middle lobe and left lingula", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5884/train_5884_a/train_5884_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5884_a_2.nii.gz", "disease_mask": "seg_rxg/train_5884_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5935/train_5935_a/train_5935_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5935_a_2.nii.gz", "disease_mask": "seg_rxg/train_5935_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Paraseptal emphysema more prominent in the right lung apex", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_5935/train_5935_a/train_5935_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5935_a_2.nii.gz", "disease_mask": "seg_rxg/train_5935_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis extending to the fissure in the posterobasal and laterobasal segments of the left lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5935/train_5935_a/train_5935_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5935_a_2.nii.gz", "disease_mask": "seg_rxg/train_5935_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Scarring changes in both lung apices, more prominent on the right", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5935/train_5935_a/train_5935_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5935_a_2.nii.gz", "disease_mask": "seg_rxg/train_5935_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subpleural nodule measuring 8 mm in diameter in the posterobasal segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5935/train_5935_a/train_5935_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5935_a_2.nii.gz", "disease_mask": "seg_rxg/train_5935_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple nonspecific nodules in both lungs, largest measuring 5 mm in the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7047/train_7047_a/train_7047_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7047_a_2.nii.gz", "disease_mask": "seg_rxg/train_7047_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Minimal scarring changes in the apex of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7047/train_7047_a/train_7047_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7047_a_2.nii.gz", "disease_mask": "seg_rxg/train_7047_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation in the peripheral area of the right lower lobe superior segment with surrounding ground glass opacity", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7047/train_7047_a/train_7047_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7047_a_2.nii.gz", "disease_mask": "seg_rxg/train_7047_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules in the right lower lobe superior segment and the left upper lobe apicoposterior segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7047/train_7047_a/train_7047_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7047_a_2.nii.gz", "disease_mask": "seg_rxg/train_7047_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities in the right lower lobe superior segment and the left upper lobe apicoposterior segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5485/train_5485_a/train_5485_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5485_a_2.nii.gz", "disease_mask": "seg_rxg/train_5485_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "7 mm nodular lesion at the basal level of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5485/train_5485_a/train_5485_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5485_a_2.nii.gz", "disease_mask": "seg_rxg/train_5485_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground glass opacity with a halo sign surrounding the 7 mm nodular lesion at the basal level of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7612/train_7612_a/train_7612_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7612_a_2.nii.gz", "disease_mask": "seg_rxg/train_7612_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Calcified nodule measuring 5 mm in the lateral basal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7612/train_7612_a/train_7612_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7612_a_2.nii.gz", "disease_mask": "seg_rxg/train_7612_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "A few subpleural nodules in both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6112/train_6112_a/train_6112_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6112_a_2.nii.gz", "disease_mask": "seg_rxg/train_6112_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Bilateral peribronchial thickening", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_6112/train_6112_a/train_6112_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6112_a_2.nii.gz", "disease_mask": "seg_rxg/train_6112_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchiectasis", "disease_findings": "Bilateral mild bronchiectatic changes", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_7296/train_7296_a/train_7296_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7296_a_2.nii.gz", "disease_mask": "seg_rxg/train_7296_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground glass opacities in the mediobasal and laterobasal segments of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7296/train_7296_a/train_7296_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7296_a_2.nii.gz", "disease_mask": "seg_rxg/train_7296_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground glass opacities in the laterobasal and posterobasal segments of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7296/train_7296_a/train_7296_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7296_a_2.nii.gz", "disease_mask": "seg_rxg/train_7296_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Pleuroparenchymal densities in the inferior lingular segment of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7296/train_7296_a/train_7296_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7296_a_2.nii.gz", "disease_mask": "seg_rxg/train_7296_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodular pleuroparenchymal density in the left lung lingular segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6687/train_6687_b/train_6687_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6687_b_2.nii.gz", "disease_mask": "seg_rxg/train_6687_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectasis in the central parts of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_6687/train_6687_b/train_6687_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6687_b_2.nii.gz", "disease_mask": "seg_rxg/train_6687_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchiectasis", "disease_findings": "Bronchiectasis more prominent in the left upper lobe lingular segment", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_6687/train_6687_b/train_6687_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6687_b_2.nii.gz", "disease_mask": "seg_rxg/train_6687_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchial wall thickening", "disease_findings": "Peribronchial thickening in the left upper lobe lingular segment", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_6687/train_6687_b/train_6687_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6687_b_2.nii.gz", "disease_mask": "seg_rxg/train_6687_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Minimal volume loss in the left upper lobe lingular segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6687/train_6687_b/train_6687_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6687_b_2.nii.gz", "disease_mask": "seg_rxg/train_6687_b_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Findings consistent with scarring in both lung apices", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6687/train_6687_b/train_6687_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6687_b_2.nii.gz", "disease_mask": "seg_rxg/train_6687_b_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodules in both lungs, many of which are calcified", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5408/train_5408_a/train_5408_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5408_a_2.nii.gz", "disease_mask": "seg_rxg/train_5408_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Several calcified nonspecific parenchymal nodules in the upper lobes of both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6723/train_6723_a/train_6723_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6723_a_2.nii.gz", "disease_mask": "seg_rxg/train_6723_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectasis in the central part of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_6723/train_6723_a/train_6723_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6723_a_2.nii.gz", "disease_mask": "seg_rxg/train_6723_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_6723/train_6723_a/train_6723_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6723_a_2.nii.gz", "disease_mask": "seg_rxg/train_6723_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7254/train_7254_a/train_7254_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7254_a_2.nii.gz", "disease_mask": "seg_rxg/train_7254_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground-glass opacities located more peripherally in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7254/train_7254_a/train_7254_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7254_a_2.nii.gz", "disease_mask": "seg_rxg/train_7254_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Crazy paving patterns located more peripherally in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7866/train_7866_a/train_7866_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7866_a_2.nii.gz", "disease_mask": "seg_rxg/train_7866_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Several nonspecific subcentimeter nodules in both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_5818/train_5818_a/train_5818_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5818_a_2.nii.gz", "disease_mask": "seg_rxg/train_5818_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion measuring 7.0 cm at its thickest point on the right", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5818/train_5818_a/train_5818_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5818_a_2.nii.gz", "disease_mask": "seg_rxg/train_5818_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the lower lobes of both lungs adjacent to the pleural effusion", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5818/train_5818_a/train_5818_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5818_a_2.nii.gz", "disease_mask": "seg_rxg/train_5818_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Septal thickening", "disease_findings": "Interlobular septal thickening in both lungs, more prominently in the upper lobes", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_5818/train_5818_a/train_5818_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5818_a_2.nii.gz", "disease_mask": "seg_rxg/train_5818_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in both lungs, more prominently in the upper lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6649/train_6649_a/train_6649_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6649_a_2.nii.gz", "disease_mask": "seg_rxg/train_6649_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Ground-glass density consolidations in both lungs extending to the subpleural distance", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6649/train_6649_a/train_6649_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6649_a_2.nii.gz", "disease_mask": "seg_rxg/train_6649_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Ground-glass density consolidations more prominent in the peripheral lung parenchyma", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6649/train_6649_a/train_6649_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6649_a_2.nii.gz", "disease_mask": "seg_rxg/train_6649_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "CT Halo sign in a few consolidations in the right lower lobe basal segment and middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7606/train_7606_a/train_7606_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7606_a_2.nii.gz", "disease_mask": "seg_rxg/train_7606_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Faintly circumscribed, barely distinguishable ground glass opacity in the mediobasal area of the superior segment of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5628/train_5628_a/train_5628_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5628_a_2.nii.gz", "disease_mask": "seg_rxg/train_5628_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Focal nodular ground glass opacity in the paramediastinal area in the superior segment of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7737/train_7737_b/train_7737_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7737_b_2.nii.gz", "disease_mask": "seg_rxg/train_7737_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation in the lower lobes of both lungs, especially in the posterobasal segments", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7737/train_7737_b/train_7737_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7737_b_2.nii.gz", "disease_mask": "seg_rxg/train_7737_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the lower lobes of both lungs, especially in the posterobasal segments", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7737/train_7737_b/train_7737_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7737_b_2.nii.gz", "disease_mask": "seg_rxg/train_7737_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pleural effusion or thickening", "disease_findings": "Bilateral minimal pleural effusion", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_5907/train_5907_a/train_5907_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5907_a_2.nii.gz", "disease_mask": "seg_rxg/train_5907_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Two nonspecific nodules, each measuring 4 mm, in the lateral segment of the right middle lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5907/train_5907_a/train_5907_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5907_a_2.nii.gz", "disease_mask": "seg_rxg/train_5907_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Two nonspecific nodules, each measuring 4 mm, in the superior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6153/train_6153_a/train_6153_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6153_a_2.nii.gz", "disease_mask": "seg_rxg/train_6153_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Pleuroparenchymal scarring densities with calcification in the bilateral upper lobe apicoposterior segments, more prominent on the left", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6153/train_6153_a/train_6153_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6153_a_2.nii.gz", "disease_mask": "seg_rxg/train_6153_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchiectasis", "disease_findings": "Bronchial dilation in both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_6153/train_6153_a/train_6153_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6153_a_2.nii.gz", "disease_mask": "seg_rxg/train_6153_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodule, possibly a lymph node, in the major fissure of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6153/train_6153_a/train_6153_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6153_a_2.nii.gz", "disease_mask": "seg_rxg/train_6153_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6153/train_6153_a/train_6153_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6153_a_2.nii.gz", "disease_mask": "seg_rxg/train_6153_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Micronodules", "disease_findings": "Multiple calcified nodules in both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6954/train_6954_a/train_6954_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6954_a_2.nii.gz", "disease_mask": "seg_rxg/train_6954_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "10 mm nodular ground glass opacity in the posterobasal segment of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6954/train_6954_a/train_6954_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6954_a_2.nii.gz", "disease_mask": "seg_rxg/train_6954_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Fibrotic changes in the apices of both upper lobes", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_6954/train_6954_a/train_6954_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6954_a_2.nii.gz", "disease_mask": "seg_rxg/train_6954_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Fibrotic band in the right middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5198/train_5198_b/train_5198_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5198_b_2.nii.gz", "disease_mask": "seg_rxg/train_5198_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Diffuse low-density centriacinar nodules in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_5640/train_5640_c/train_5640_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5640_c_2.nii.gz", "disease_mask": "seg_rxg/train_5640_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Drainage catheter placed in the consolidation and effusion area containing air densities in the lower part of the right hemithorax", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5640/train_5640_c/train_5640_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5640_c_2.nii.gz", "disease_mask": "seg_rxg/train_5640_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Stable consolidation and bronchopleural fistula anterior to the effusion", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6509/train_6509_a/train_6509_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6509_a_2.nii.gz", "disease_mask": "seg_rxg/train_6509_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Peripheral and central consolidations in the upper lobe of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6509/train_6509_a/train_6509_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6509_a_2.nii.gz", "disease_mask": "seg_rxg/train_6509_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Minimal ground-glass opacities in the upper lobe of the left lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6509/train_6509_a/train_6509_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6509_a_2.nii.gz", "disease_mask": "seg_rxg/train_6509_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Micronodules", "disease_findings": "Centrilobular nodules in the upper lobe of the left lung", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6813/train_6813_a/train_6813_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6813_a_2.nii.gz", "disease_mask": "seg_rxg/train_6813_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Multilobar, peripheral, subpleural, dorsal consolidation in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6813/train_6813_a/train_6813_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6813_a_2.nii.gz", "disease_mask": "seg_rxg/train_6813_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Areas of increased ground glass density in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6039/train_6039_b/train_6039_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6039_b_2.nii.gz", "disease_mask": "seg_rxg/train_6039_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Minimal effusion measuring 8 mm in diameter in the deepest part of the left pleural space", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_6039/train_6039_b/train_6039_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6039_b_2.nii.gz", "disease_mask": "seg_rxg/train_6039_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Multisegmental, mostly peripheral subpleural ground-glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6039/train_6039_b/train_6039_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6039_b_2.nii.gz", "disease_mask": "seg_rxg/train_6039_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of consolidation in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7004/train_7004_a/train_7004_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7004_a_2.nii.gz", "disease_mask": "seg_rxg/train_7004_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodule in the anterior segment of the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7683/train_7683_a/train_7683_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7683_a_2.nii.gz", "disease_mask": "seg_rxg/train_7683_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Nodule measuring 5 mm with a ground-glass halo in the apicoposterior segment of the left upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7683/train_7683_a/train_7683_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7683_a_2.nii.gz", "disease_mask": "seg_rxg/train_7683_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Scarring in the middle lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7683/train_7683_a/train_7683_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7683_a_2.nii.gz", "disease_mask": "seg_rxg/train_7683_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Scarring in the inferior lingular segment of the left lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6377/train_6377_b/train_6377_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6377_b_2.nii.gz", "disease_mask": "seg_rxg/train_6377_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_6377/train_6377_b/train_6377_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6377_b_2.nii.gz", "disease_mask": "seg_rxg/train_6377_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Scarring in the apices of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6377/train_6377_b/train_6377_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6377_b_2.nii.gz", "disease_mask": "seg_rxg/train_6377_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectatic changes centrally", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_12251/train_12251_b/train_12251_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12251_b_2.nii.gz", "disease_mask": "seg_rxg/train_12251_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Mild patchy ground-glass opacities in both lungs, more prominently in the right middle lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_10388/train_10388_a/train_10388_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10388_a_2.nii.gz", "disease_mask": "seg_rxg/train_10388_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Atelectatic change in the form of a linear band at the posterobasal level of the left lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10330/train_10330_a/train_10330_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10330_a_2.nii.gz", "disease_mask": "seg_rxg/train_10330_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific parenchymal nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_12915/train_12915_b/train_12915_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12915_b_2.nii.gz", "disease_mask": "seg_rxg/train_12915_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Extensive ground-glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_10865/train_10865_a/train_10865_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10865_a_2.nii.gz", "disease_mask": "seg_rxg/train_10865_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Pleuroparenchymal band formations in all lobes and segments of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10865/train_10865_a/train_10865_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10865_a_2.nii.gz", "disease_mask": "seg_rxg/train_10865_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchiectasis", "disease_findings": "Traction bronchiectasis in the lower lobes of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_12460/train_12460_c/train_12460_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12460_c_2.nii.gz", "disease_mask": "seg_rxg/train_12460_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodule in the anterior upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10180/train_10180_a/train_10180_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10180_a_2.nii.gz", "disease_mask": "seg_rxg/train_10180_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodule in the apical region of the right upper lobe (series 2, image 115)", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10581/train_10581_a/train_10581_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10581_a_2.nii.gz", "disease_mask": "seg_rxg/train_10581_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the anteromediobasal segment of the lower lobe of the left lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10932/train_10932_a/train_10932_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10932_a_2.nii.gz", "disease_mask": "seg_rxg/train_10932_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Scattered and patchy ground-glass opacities in both lungs, particularly in the subpleural areas", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_12030/train_12030_b/train_12030_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12030_b_2.nii.gz", "disease_mask": "seg_rxg/train_12030_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Minimal peribronchial thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_12030/train_12030_b/train_12030_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12030_b_2.nii.gz", "disease_mask": "seg_rxg/train_12030_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_10937/train_10937_a/train_10937_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10937_a_2.nii.gz", "disease_mask": "seg_rxg/train_10937_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Mild bronchiectatic changes in the central regions of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_10937/train_10937_a/train_10937_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10937_a_2.nii.gz", "disease_mask": "seg_rxg/train_10937_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchial wall thickening", "disease_findings": "Minimal peribronchial thickening in the central regions of both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_10937/train_10937_a/train_10937_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10937_a_2.nii.gz", "disease_mask": "seg_rxg/train_10937_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "5 mm calcified parenchymal nodule in the peripheral subpleural area of the apicoposterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_12317/train_12317_a/train_12317_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12317_a_2.nii.gz", "disease_mask": "seg_rxg/train_12317_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass opacities in the peripheral subpleural area of the posterobasal segments of both lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11828/train_11828_a/train_11828_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11828_a_2.nii.gz", "disease_mask": "seg_rxg/train_11828_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear subsegmental atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11828/train_11828_a/train_11828_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11828_a_2.nii.gz", "disease_mask": "seg_rxg/train_11828_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific pulmonary nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10826/train_10826_a/train_10826_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10826_a_2.nii.gz", "disease_mask": "seg_rxg/train_10826_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Prominent emphysematous changes in the upper lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_10826/train_10826_a/train_10826_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10826_a_2.nii.gz", "disease_mask": "seg_rxg/train_10826_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple parenchymal nodules in both lungs, largest measuring 6 mm in the middle lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11889/train_11889_a/train_11889_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11889_a_2.nii.gz", "disease_mask": "seg_rxg/train_11889_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_11889/train_11889_a/train_11889_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11889_a_2.nii.gz", "disease_mask": "seg_rxg/train_11889_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the right middle lobe medial segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11889/train_11889_a/train_11889_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11889_a_2.nii.gz", "disease_mask": "seg_rxg/train_11889_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the left upper lobe lingular segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10802/train_10802_a/train_10802_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10802_a_2.nii.gz", "disease_mask": "seg_rxg/train_10802_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific pulmonary nodules in the posterobasal parts of the lower lobes of both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10802/train_10802_a/train_10802_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10802_a_2.nii.gz", "disease_mask": "seg_rxg/train_10802_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in the upper lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_10802/train_10802_a/train_10802_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10802_a_2.nii.gz", "disease_mask": "seg_rxg/train_10802_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11249/train_11249_a/train_11249_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11249_a_2.nii.gz", "disease_mask": "seg_rxg/train_11249_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in the lower lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_10043/train_10043_b/train_10043_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10043_b_2.nii.gz", "disease_mask": "seg_rxg/train_10043_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Bilateral pleural effusion", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_12553/train_12553_a/train_12553_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12553_a_2.nii.gz", "disease_mask": "seg_rxg/train_12553_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Increased density consistent with scarring in the middle lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12553/train_12553_a/train_12553_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12553_a_2.nii.gz", "disease_mask": "seg_rxg/train_12553_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Increased density consistent with scarring in the inferior lingular segment of the left lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11063/train_11063_a/train_11063_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11063_a_2.nii.gz", "disease_mask": "seg_rxg/train_11063_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral and centrally located ground glass opacities in the upper and lower lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11063/train_11063_a/train_11063_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11063_a_2.nii.gz", "disease_mask": "seg_rxg/train_11063_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Round-shaped ground glass opacities", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_10903/train_10903_a/train_10903_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10903_a_2.nii.gz", "disease_mask": "seg_rxg/train_10903_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10903/train_10903_a/train_10903_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10903_a_2.nii.gz", "disease_mask": "seg_rxg/train_10903_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Thin fibrotic bands in the middle lobe on the right", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10903/train_10903_a/train_10903_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10903_a_2.nii.gz", "disease_mask": "seg_rxg/train_10903_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Thin fibrotic bands in the lingula on the left", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10941/train_10941_b/train_10941_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10941_b_2.nii.gz", "disease_mask": "seg_rxg/train_10941_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Increased pleuroparenchymal density consistent with scarring in the apices of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10941/train_10941_b/train_10941_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10941_b_2.nii.gz", "disease_mask": "seg_rxg/train_10941_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Mild emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_10179/train_10179_a/train_10179_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10179_a_2.nii.gz", "disease_mask": "seg_rxg/train_10179_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground-glass opacities in both lungs, predominantly in the subpleural areas", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_10061/train_10061_a/train_10061_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10061_a_2.nii.gz", "disease_mask": "seg_rxg/train_10061_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Central and peripheral ground-glass opacities in the upper and lower lobes of both lungs and the middle lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_12655/train_12655_a/train_12655_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12655_a_2.nii.gz", "disease_mask": "seg_rxg/train_12655_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Densities consistent with scarring on the pleural surfaces of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11323/train_11323_a/train_11323_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11323_a_2.nii.gz", "disease_mask": "seg_rxg/train_11323_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_11165/train_11165_a/train_11165_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11165_a_2.nii.gz", "disease_mask": "seg_rxg/train_11165_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Mild paraseptal emphysematous changes at the left apical level", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_11997/train_11997_a/train_11997_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11997_a_2.nii.gz", "disease_mask": "seg_rxg/train_11997_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Dependent density increases in the lower lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_11997/train_11997_a/train_11997_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11997_a_2.nii.gz", "disease_mask": "seg_rxg/train_11997_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass areas in the mediobasal segment of the lower lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11997/train_11997_a/train_11997_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11997_a_2.nii.gz", "disease_mask": "seg_rxg/train_11997_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass areas in the medial part of the lower lobe superior segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11997/train_11997_a/train_11997_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11997_a_2.nii.gz", "disease_mask": "seg_rxg/train_11997_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass areas in the anterior segments of the upper lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11997/train_11997_a/train_11997_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11997_a_2.nii.gz", "disease_mask": "seg_rxg/train_11997_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several nonspecific nodules measuring 2-3 mm in diameter in the anterior segment of the upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10185/train_10185_a/train_10185_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10185_a_2.nii.gz", "disease_mask": "seg_rxg/train_10185_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities in the posterior lower lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_12459/train_12459_a/train_12459_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12459_a_2.nii.gz", "disease_mask": "seg_rxg/train_12459_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific stable nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10502/train_10502_b/train_10502_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10502_b_2.nii.gz", "disease_mask": "seg_rxg/train_10502_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Mild emphysematous changes in the upper lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_11903/train_11903_a/train_11903_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11903_a_2.nii.gz", "disease_mask": "seg_rxg/train_11903_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple subpleural nodules up to 8 mm in size in the posterior lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11903/train_11903_a/train_11903_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11903_a_2.nii.gz", "disease_mask": "seg_rxg/train_11903_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest nodules located in the upper lobes of both lungs, especially at the apical levels", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11496/train_11496_b/train_11496_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11496_b_2.nii.gz", "disease_mask": "seg_rxg/train_11496_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacity with indistinct borders in the peripheral area of the upper lobe of the left lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_10417/train_10417_a/train_10417_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10417_a_2.nii.gz", "disease_mask": "seg_rxg/train_10417_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Scattered patchy and nodular ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_12179/train_12179_a/train_12179_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12179_a_2.nii.gz", "disease_mask": "seg_rxg/train_12179_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities in the superior segment of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_12179/train_12179_a/train_12179_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12179_a_2.nii.gz", "disease_mask": "seg_rxg/train_12179_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Crazy paving appearance in the superior segment of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_10008/train_10008_a/train_10008_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10008_a_2.nii.gz", "disease_mask": "seg_rxg/train_10008_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Patchy areas with air signs and vascular enlargement in the upper lobe of the left lung at the level of the superior lingula", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12397/train_12397_a/train_12397_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12397_a_2.nii.gz", "disease_mask": "seg_rxg/train_12397_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Nodular and patchy ground-glass opacities in both lungs, predominantly subpleural", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11428/train_11428_b/train_11428_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11428_b_2.nii.gz", "disease_mask": "seg_rxg/train_11428_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "One or two subcentimeter nonspecific nodules in the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11164/train_11164_b/train_11164_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11164_b_2.nii.gz", "disease_mask": "seg_rxg/train_11164_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Three paravertebral and centrally located lesions in the posterior and lateral segment of the right lower lobe, measuring 18 mm and 10 mm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11164/train_11164_b/train_11164_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11164_b_2.nii.gz", "disease_mask": "seg_rxg/train_11164_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Anterior segment lesion in the right upper lobe posteriorly in the paramediastinal area", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11164/train_11164_b/train_11164_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11164_b_2.nii.gz", "disease_mask": "seg_rxg/train_11164_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Apical fibrotic recessions and calcific foci in the upper lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11164/train_11164_b/train_11164_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11164_b_2.nii.gz", "disease_mask": "seg_rxg/train_11164_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Changes secondary to TB in the upper lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12039/train_12039_a/train_12039_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12039_a_2.nii.gz", "disease_mask": "seg_rxg/train_12039_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Central and peripheral ground-glass opacities with a crazy paving pattern and irregular borders in the right middle lobe and both lower lobe basal segments", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_12039/train_12039_a/train_12039_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12039_a_2.nii.gz", "disease_mask": "seg_rxg/train_12039_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Scarring and atelectasis in the left upper lobe lingula", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_10253/train_10253_b/train_10253_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10253_b_2.nii.gz", "disease_mask": "seg_rxg/train_10253_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Bilateral pleural effusion measuring 3.3 cm at its thickest point on the right", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_10253/train_10253_b/train_10253_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10253_b_2.nii.gz", "disease_mask": "seg_rxg/train_10253_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the lower lobes of both lungs adjacent to the pleural effusion, especially in the basal segments of the right lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_10253/train_10253_b/train_10253_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10253_b_2.nii.gz", "disease_mask": "seg_rxg/train_10253_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_10689/train_10689_a/train_10689_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10689_a_2.nii.gz", "disease_mask": "seg_rxg/train_10689_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10981/train_10981_a/train_10981_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10981_a_2.nii.gz", "disease_mask": "seg_rxg/train_10981_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Pulmonary nodules in both lungs measuring up to 7.6 mm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10981/train_10981_a/train_10981_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10981_a_2.nii.gz", "disease_mask": "seg_rxg/train_10981_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest pulmonary nodule superimposed on the vascular structure in the middle lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11412/train_11412_a/train_11412_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11412_a_2.nii.gz", "disease_mask": "seg_rxg/train_11412_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodule in the anterior upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11412/train_11412_a/train_11412_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11412_a_2.nii.gz", "disease_mask": "seg_rxg/train_11412_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Minimal dependent densities in the posterobasal areas of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_10784/train_10784_a/train_10784_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10784_a_2.nii.gz", "disease_mask": "seg_rxg/train_10784_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the lower lobe of the left lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10353/train_10353_a/train_10353_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10353_a_2.nii.gz", "disease_mask": "seg_rxg/train_10353_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Subpleural patchy ground glass opacities in both lungs, more prominently in the lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_10106/train_10106_a/train_10106_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10106_a_2.nii.gz", "disease_mask": "seg_rxg/train_10106_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_12475/train_12475_a/train_12475_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12475_a_2.nii.gz", "disease_mask": "seg_rxg/train_12475_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear subsegmental atelectasis in the superior segment of the left lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12475/train_12475_a/train_12475_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12475_a_2.nii.gz", "disease_mask": "seg_rxg/train_12475_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear subsegmental atelectasis in the inferior lingular segment of the left upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12475/train_12475_a/train_12475_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12475_a_2.nii.gz", "disease_mask": "seg_rxg/train_12475_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear subsegmental atelectasis in the medial segment of the right middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12588/train_12588_b/train_12588_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12588_b_2.nii.gz", "disease_mask": "seg_rxg/train_12588_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground glass densities in the upper lobe posterior segment of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_12588/train_12588_b/train_12588_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12588_b_2.nii.gz", "disease_mask": "seg_rxg/train_12588_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion in both hemithoraces", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_12588/train_12588_b/train_12588_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12588_b_2.nii.gz", "disease_mask": "seg_rxg/train_12588_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion thickness approximately 4.5 cm on the right", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_12588/train_12588_b/train_12588_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12588_b_2.nii.gz", "disease_mask": "seg_rxg/train_12588_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion thickness approximately 1.5 cm on the left", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_10445/train_10445_c/train_10445_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10445_c_2.nii.gz", "disease_mask": "seg_rxg/train_10445_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Scarring in both lung apices", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10445/train_10445_c/train_10445_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10445_c_2.nii.gz", "disease_mask": "seg_rxg/train_10445_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lung apices", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_10445/train_10445_c/train_10445_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10445_c_2.nii.gz", "disease_mask": "seg_rxg/train_10445_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchiectasis", "disease_findings": "Bronchiectatic changes in both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_10445/train_10445_c/train_10445_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10445_c_2.nii.gz", "disease_mask": "seg_rxg/train_10445_c_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Bronchial wall thickening", "disease_findings": "Minimal peribronchial thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_12430/train_12430_a/train_12430_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12430_a_2.nii.gz", "disease_mask": "seg_rxg/train_12430_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Extensive consolidation with air bronchograms in the right lung lower lobe superior segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12430/train_12430_a/train_12430_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12430_a_2.nii.gz", "disease_mask": "seg_rxg/train_12430_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Extensive consolidation with air bronchograms in the right lung lower lobe lateral segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12430/train_12430_a/train_12430_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12430_a_2.nii.gz", "disease_mask": "seg_rxg/train_12430_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Extensive consolidation with air bronchograms in the right lung lower lobe posterobasal segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12430/train_12430_a/train_12430_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12430_a_2.nii.gz", "disease_mask": "seg_rxg/train_12430_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Micronodules", "disease_findings": "Subcentimeter nodular consolidation areas with a tree-in-bud appearance in the left lung lower lobe posterobasal segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_12277/train_12277_a/train_12277_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12277_a_2.nii.gz", "disease_mask": "seg_rxg/train_12277_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Pleuroparenchymal scarring densities in the anterior segment of the right upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12277/train_12277_a/train_12277_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12277_a_2.nii.gz", "disease_mask": "seg_rxg/train_12277_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Calcified nodule in the anterior segment of the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_12277/train_12277_a/train_12277_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12277_a_2.nii.gz", "disease_mask": "seg_rxg/train_12277_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Pleuroparenchymal scarring densities in the right middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12277/train_12277_a/train_12277_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12277_a_2.nii.gz", "disease_mask": "seg_rxg/train_12277_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific nodule measuring 2.5 mm in the anterior segment of the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_12423/train_12423_a/train_12423_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12423_a_2.nii.gz", "disease_mask": "seg_rxg/train_12423_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Minimal band-like fibrotic changes extending to the pleura in the posterobasal segment of the right lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12423/train_12423_a/train_12423_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12423_a_2.nii.gz", "disease_mask": "seg_rxg/train_12423_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral ground-glass nodules in the posterobasal segment of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11463/train_11463_c/train_11463_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11463_c_2.nii.gz", "disease_mask": "seg_rxg/train_11463_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_12558/train_12558_a/train_12558_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12558_a_2.nii.gz", "disease_mask": "seg_rxg/train_12558_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of increased density consistent with consolidation in the posterobasal segment of the left lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12558/train_12558_a/train_12558_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12558_a_2.nii.gz", "disease_mask": "seg_rxg/train_12558_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of increased density consistent with consolidation in the superior segment of the left lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12558/train_12558_a/train_12558_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12558_a_2.nii.gz", "disease_mask": "seg_rxg/train_12558_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of increased density consistent with consolidation in the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12558/train_12558_a/train_12558_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12558_a_2.nii.gz", "disease_mask": "seg_rxg/train_12558_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of increased density consistent with consolidation in the lower lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12558/train_12558_a/train_12558_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12558_a_2.nii.gz", "disease_mask": "seg_rxg/train_12558_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several nonspecific nodules in both lungs, the largest measuring 4 mm in the posterior segment of the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11142/train_11142_a/train_11142_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11142_a_2.nii.gz", "disease_mask": "seg_rxg/train_11142_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Nodular ground glass densities with patchy halo markings in the lower lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11142/train_11142_a/train_11142_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11142_a_2.nii.gz", "disease_mask": "seg_rxg/train_11142_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Nodular ground glass densities with patchy halo markings in the upper lobe of the left lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11507/train_11507_a/train_11507_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11507_a_2.nii.gz", "disease_mask": "seg_rxg/train_11507_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Increased density consistent with scarring in the middle lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11507/train_11507_a/train_11507_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11507_a_2.nii.gz", "disease_mask": "seg_rxg/train_11507_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Increased density consistent with scarring in the inferior lingular segment of the left lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12301/train_12301_a/train_12301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12301_a_2.nii.gz", "disease_mask": "seg_rxg/train_12301_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Subcentimeter ground-glass nodules in a focal area in the posterior segment of the right upper lobe, adjacent to the fissure", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11995/train_11995_a/train_11995_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11995_a_2.nii.gz", "disease_mask": "seg_rxg/train_11995_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities surrounding the consolidation area", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11995/train_11995_a/train_11995_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11995_a_2.nii.gz", "disease_mask": "seg_rxg/train_11995_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Increase in pleural thickness, including calcification, adjacent to the consolidation area", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_12229/train_12229_a/train_12229_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12229_a_2.nii.gz", "disease_mask": "seg_rxg/train_12229_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10445/train_10445_e/train_10445_e_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10445_e_2.nii.gz", "disease_mask": "seg_rxg/train_10445_e_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Paraseptal emphysematous changes in the apical segments of the upper lobes of both lungs, more prominently on the right", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_10741/train_10741_a/train_10741_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10741_a_2.nii.gz", "disease_mask": "seg_rxg/train_10741_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific nodules in both lungs, measuring 7x3 mm in the posterior segment fissure of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10741/train_10741_a/train_10741_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10741_a_2.nii.gz", "disease_mask": "seg_rxg/train_10741_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific nodules in both lungs, measuring 8x3.5 mm in the major fissure of the right middle lobe lateral segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10427/train_10427_b/train_10427_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10427_b_2.nii.gz", "disease_mask": "seg_rxg/train_10427_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few nonspecific parenchymal nodules under 5 mm in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11240/train_11240_c/train_11240_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11240_c_2.nii.gz", "disease_mask": "seg_rxg/train_11240_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities in both lungs, particularly in the subpleural areas and more prominently in the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_10882/train_10882_a/train_10882_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10882_a_2.nii.gz", "disease_mask": "seg_rxg/train_10882_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Cylindrical bronchiectasis at the basal level of the left lower lobe", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_10882/train_10882_a/train_10882_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10882_a_2.nii.gz", "disease_mask": "seg_rxg/train_10882_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "One or two subpleural nodules in the lower lobe of the left lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10410/train_10410_a/train_10410_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10410_a_2.nii.gz", "disease_mask": "seg_rxg/train_10410_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Intermittent linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11284/train_11284_a/train_11284_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11284_a_2.nii.gz", "disease_mask": "seg_rxg/train_11284_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Diffuse ground-glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11284/train_11284_a/train_11284_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11284_a_2.nii.gz", "disease_mask": "seg_rxg/train_11284_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Posterior predominant opacities in the lower lobes", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_11437/train_11437_a/train_11437_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11437_a_2.nii.gz", "disease_mask": "seg_rxg/train_11437_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Multilobar areas of ground-glass opacity in the peripheral subpleural regions of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11437/train_11437_a/train_11437_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11437_a_2.nii.gz", "disease_mask": "seg_rxg/train_11437_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Increased density consistent with consolidation in the peripheral subpleural regions of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_11437/train_11437_a/train_11437_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11437_a_2.nii.gz", "disease_mask": "seg_rxg/train_11437_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Parenchymal changes consistent with scarring in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10380/train_10380_a/train_10380_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10380_a_2.nii.gz", "disease_mask": "seg_rxg/train_10380_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities extending to the pleura in the lateral segment of the right middle lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_10380/train_10380_a/train_10380_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10380_a_2.nii.gz", "disease_mask": "seg_rxg/train_10380_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Minimal subpleural focal consolidations in the lateral segment of the right middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_10380/train_10380_a/train_10380_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10380_a_2.nii.gz", "disease_mask": "seg_rxg/train_10380_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities extending to the pleura in the anterior segment of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_10380/train_10380_a/train_10380_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10380_a_2.nii.gz", "disease_mask": "seg_rxg/train_10380_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Minimal subpleural focal consolidations in the anterior segment of the left lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_10380/train_10380_a/train_10380_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10380_a_2.nii.gz", "disease_mask": "seg_rxg/train_10380_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Calcified nodules consistent with scarring in the posterior segment of the right upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10682/train_10682_a/train_10682_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10682_a_2.nii.gz", "disease_mask": "seg_rxg/train_10682_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few subcentimeter nodules, some calcified and some solid, in both lung parenchyma", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_12105/train_12105_b/train_12105_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12105_b_2.nii.gz", "disease_mask": "seg_rxg/train_12105_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Peripheral and central consolidations with ground glass opacities in the upper, middle, and lower lobes of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12105/train_12105_b/train_12105_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12105_b_2.nii.gz", "disease_mask": "seg_rxg/train_12105_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidations in the apicoposterior and lingular segments of the left upper lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12105/train_12105_b/train_12105_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12105_b_2.nii.gz", "disease_mask": "seg_rxg/train_12105_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidations in the superior and basal segments of the left lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12105/train_12105_b/train_12105_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12105_b_2.nii.gz", "disease_mask": "seg_rxg/train_12105_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_11848/train_11848_a/train_11848_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11848_a_2.nii.gz", "disease_mask": "seg_rxg/train_11848_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Increased density consistent with scarring in the inferior lingular segment of the left lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11848/train_11848_a/train_11848_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11848_a_2.nii.gz", "disease_mask": "seg_rxg/train_11848_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subpleural 5 mm nonspecific parenchymal nodule in the middle lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10973/train_10973_b/train_10973_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10973_b_2.nii.gz", "disease_mask": "seg_rxg/train_10973_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear areas of atelectasis in the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12610/train_12610_a/train_12610_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12610_a_2.nii.gz", "disease_mask": "seg_rxg/train_12610_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Pleuroparenchymal fibroatelectasis and scarring in the right middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12610/train_12610_a/train_12610_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12610_a_2.nii.gz", "disease_mask": "seg_rxg/train_12610_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Pleuroparenchymal fibroatelectasis and scarring in the inferior lingular segment of the left upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12610/train_12610_a/train_12610_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12610_a_2.nii.gz", "disease_mask": "seg_rxg/train_12610_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Micronodules", "disease_findings": "Focal centriacinar nodules in the basal segment of the left lower lobe", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_10560/train_10560_a/train_10560_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10560_a_2.nii.gz", "disease_mask": "seg_rxg/train_10560_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Mild scarring at the apical level", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10560/train_10560_a/train_10560_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10560_a_2.nii.gz", "disease_mask": "seg_rxg/train_10560_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "5x3 mm nodule in dorsal subpleural area of superior segment of right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_10560/train_10560_a/train_10560_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10560_a_2.nii.gz", "disease_mask": "seg_rxg/train_10560_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Micronodules", "disease_findings": "Branching pattern with faint nodules in posterior segment of right upper lobe", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_11573/train_11573_d/train_11573_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11573_d_2.nii.gz", "disease_mask": "seg_rxg/train_11573_d_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral and centrally located ground-glass opacities in the upper and lower lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11573/train_11573_d/train_11573_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11573_d_2.nii.gz", "disease_mask": "seg_rxg/train_11573_d_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral and centrally located ground-glass opacities in the middle lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_10926/train_10926_a/train_10926_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10926_a_2.nii.gz", "disease_mask": "seg_rxg/train_10926_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion with a depth of 15 mm on the right", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_10926/train_10926_a/train_10926_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10926_a_2.nii.gz", "disease_mask": "seg_rxg/train_10926_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion with a depth of 10 mm on the left", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_10926/train_10926_a/train_10926_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10926_a_2.nii.gz", "disease_mask": "seg_rxg/train_10926_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific nodules in the lung parenchyma", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_12699/train_12699_b/train_12699_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12699_b_2.nii.gz", "disease_mask": "seg_rxg/train_12699_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Almost complete atelectasis in the upper lobe of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12699/train_12699_b/train_12699_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12699_b_2.nii.gz", "disease_mask": "seg_rxg/train_12699_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Pleural effusion on the left", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_10846/train_10846_b/train_10846_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10846_b_2.nii.gz", "disease_mask": "seg_rxg/train_10846_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the upper lobe of the left lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10846/train_10846_b/train_10846_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10846_b_2.nii.gz", "disease_mask": "seg_rxg/train_10846_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the lower lobe of the left lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_10846/train_10846_b/train_10846_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10846_b_2.nii.gz", "disease_mask": "seg_rxg/train_10846_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Smaller areas of atelectasis in the lower lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11182/train_11182_a/train_11182_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11182_a_2.nii.gz", "disease_mask": "seg_rxg/train_11182_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the upper lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_11182/train_11182_a/train_11182_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11182_a_2.nii.gz", "disease_mask": "seg_rxg/train_11182_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_11182/train_11182_a/train_11182_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11182_a_2.nii.gz", "disease_mask": "seg_rxg/train_11182_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the lower lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12155/train_12155_a/train_12155_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12155_a_2.nii.gz", "disease_mask": "seg_rxg/train_12155_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter nodules in both lungs, largest measuring 2.5 mm in the superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_12155/train_12155_a/train_12155_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12155_a_2.nii.gz", "disease_mask": "seg_rxg/train_12155_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Areas of linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12957/train_12957_a/train_12957_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12957_a_2.nii.gz", "disease_mask": "seg_rxg/train_12957_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Minimal pleural effusion on the right", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_12957/train_12957_a/train_12957_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12957_a_2.nii.gz", "disease_mask": "seg_rxg/train_12957_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation in the posterobasal segment of the lower lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_12957/train_12957_a/train_12957_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12957_a_2.nii.gz", "disease_mask": "seg_rxg/train_12957_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectasis in the posterobasal segment of the lower lobe of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_10634/train_10634_b/train_10634_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10634_b_2.nii.gz", "disease_mask": "seg_rxg/train_10634_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Minimal fibrotic changes in the upper lobes of both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_10634/train_10634_b/train_10634_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_10634_b_2.nii.gz", "disease_mask": "seg_rxg/train_10634_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodules measuring 3 mm in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13724/train_13724_a/train_13724_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13724_a_2.nii.gz", "disease_mask": "seg_rxg/train_13724_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13890/train_13890_a/train_13890_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13890_a_2.nii.gz", "disease_mask": "seg_rxg/train_13890_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripherally located crazy paving patterns forming nodular ground glass consolidations in the basal segments of both lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_13868/train_13868_a/train_13868_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13868_a_2.nii.gz", "disease_mask": "seg_rxg/train_13868_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacity infiltration in both lungs, increasing towards the bases", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_13804/train_13804_a/train_13804_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13804_a_2.nii.gz", "disease_mask": "seg_rxg/train_13804_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific subpleural nodule in the superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13748/train_13748_a/train_13748_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13748_a_2.nii.gz", "disease_mask": "seg_rxg/train_13748_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground-glass opacities with prominent septal thickening in the upper and lower lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_13748/train_13748_a/train_13748_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13748_a_2.nii.gz", "disease_mask": "seg_rxg/train_13748_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific parenchymal nodules, some calcified, in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13804/train_13804_b/train_13804_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13804_b_2.nii.gz", "disease_mask": "seg_rxg/train_13804_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Stable nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13661/train_13661_a/train_13661_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13661_a_2.nii.gz", "disease_mask": "seg_rxg/train_13661_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific nodules in both lung parenchyma", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13789/train_13789_a/train_13789_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13789_a_2.nii.gz", "disease_mask": "seg_rxg/train_13789_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacity in the superior segment of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_13789/train_13789_a/train_13789_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13789_a_2.nii.gz", "disease_mask": "seg_rxg/train_13789_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Minimal interlobular septal thickening in the superior segment of the right lower lobe", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_13780/train_13780_a/train_13780_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13780_a_2.nii.gz", "disease_mask": "seg_rxg/train_13780_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Paraseptal emphysematous changes in the upper lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_13780/train_13780_a/train_13780_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13780_a_2.nii.gz", "disease_mask": "seg_rxg/train_13780_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear fibrotic bands in the left lung lingular segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13780/train_13780_a/train_13780_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13780_a_2.nii.gz", "disease_mask": "seg_rxg/train_13780_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear fibrotic bands in the left lung lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13780/train_13780_a/train_13780_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13780_a_2.nii.gz", "disease_mask": "seg_rxg/train_13780_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Linear fibrotic bands in the right lung lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13780/train_13780_a/train_13780_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13780_a_2.nii.gz", "disease_mask": "seg_rxg/train_13780_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Linear fibrotic bands in the right lung middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13780/train_13780_a/train_13780_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13780_a_2.nii.gz", "disease_mask": "seg_rxg/train_13780_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Linear", "disease_findings": "Linear fibrotic bands in the right lung upper lobe anterior segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13883/train_13883_a/train_13883_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13883_a_2.nii.gz", "disease_mask": "seg_rxg/train_13883_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral subcentimeter ground glass opacities in the lower lobes and upper lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_13883/train_13883_a/train_13883_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13883_a_2.nii.gz", "disease_mask": "seg_rxg/train_13883_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Centrally located subcentimeter ground glass opacities in the lower lobes and upper lobes of both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_13880/train_13880_a/train_13880_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13880_a_2.nii.gz", "disease_mask": "seg_rxg/train_13880_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_115/train_115_a/train_115_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_115_a_2.nii.gz", "disease_mask": "seg_rxg/train_115_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs, more pronounced in the left upper lobe", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_115/train_115_a/train_115_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_115_a_2.nii.gz", "disease_mask": "seg_rxg/train_115_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_272/train_272_a/train_272_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_272_a_2.nii.gz", "disease_mask": "seg_rxg/train_272_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Scattered patches of ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_272/train_272_a/train_272_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_272_a_2.nii.gz", "disease_mask": "seg_rxg/train_272_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation present in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_240/train_240_a/train_240_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_240_a_2.nii.gz", "disease_mask": "seg_rxg/train_240_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Reticulonodular densities in the subpleural regions of both lower lobes", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_240/train_240_a/train_240_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_240_a_2.nii.gz", "disease_mask": "seg_rxg/train_240_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities in the subpleural regions of both lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_230/train_230_a/train_230_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_230_a_2.nii.gz", "disease_mask": "seg_rxg/train_230_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple subcentimeter nonspecific nodules within both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_263/train_263_h/train_263_h_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_263_h_2.nii.gz", "disease_mask": "seg_rxg/train_263_h_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Minimal pleural effusion bilaterally", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_263/train_263_h/train_263_h_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_263_h_2.nii.gz", "disease_mask": "seg_rxg/train_263_h_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Mosaic attenuation pattern throughout both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_263/train_263_h/train_263_h_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_263_h_2.nii.gz", "disease_mask": "seg_rxg/train_263_h_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchial wall thickening", "disease_findings": "Peribronchial thickening", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_263/train_263_h/train_263_h_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_263_h_2.nii.gz", "disease_mask": "seg_rxg/train_263_h_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Micronodules", "disease_findings": "Tree-in-bud nodularity in bilateral lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_263/train_263_h/train_263_h_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_263_h_2.nii.gz", "disease_mask": "seg_rxg/train_263_h_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_93/train_93_d/train_93_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_93_d_2.nii.gz", "disease_mask": "seg_rxg/train_93_d_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular consolidation in the subpleural region of the superior segment of the left lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_93/train_93_d/train_93_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_93_d_2.nii.gz", "disease_mask": "seg_rxg/train_93_d_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Surrounding ground glass opacities in the superior segment of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_197/train_197_a/train_197_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_197_a_2.nii.gz", "disease_mask": "seg_rxg/train_197_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple indeterminate subcentimeter parenchymal nodules within both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_296/train_296_a/train_296_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_296_a_2.nii.gz", "disease_mask": "seg_rxg/train_296_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground-glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_20/train_20_a/train_20_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_20_a_2.nii.gz", "disease_mask": "seg_rxg/train_20_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral and central ground glass opacities in bilateral lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_327/train_327_a/train_327_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_327_a_2.nii.gz", "disease_mask": "seg_rxg/train_327_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_327/train_327_a/train_327_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_327_a_2.nii.gz", "disease_mask": "seg_rxg/train_327_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodules in bilateral lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_107/train_107_c/train_107_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_107_c_2.nii.gz", "disease_mask": "seg_rxg/train_107_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Large consolidation in the lower lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_107/train_107_c/train_107_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_107_c_2.nii.gz", "disease_mask": "seg_rxg/train_107_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Surrounding ground glass opacities in the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_107/train_107_c/train_107_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_107_c_2.nii.gz", "disease_mask": "seg_rxg/train_107_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Scattered focal consolidation areas in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_107/train_107_c/train_107_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_107_c_2.nii.gz", "disease_mask": "seg_rxg/train_107_c_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities surrounding scattered focal consolidation areas", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_214/train_214_a/train_214_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_214_a_2.nii.gz", "disease_mask": "seg_rxg/train_214_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Pleural-based consolidation with air bronchograms in the superior and lateral portions of the right lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_214/train_214_a/train_214_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_214_a_2.nii.gz", "disease_mask": "seg_rxg/train_214_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities surrounding the consolidation area", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_214/train_214_a/train_214_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_214_a_2.nii.gz", "disease_mask": "seg_rxg/train_214_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Additional ground glass opacities in the posterobasal segments", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_214/train_214_a/train_214_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_214_a_2.nii.gz", "disease_mask": "seg_rxg/train_214_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Emphysema", "disease_findings": "Bilateral pulmonary emphysematous changes", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_214/train_214_a/train_214_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_214_a_2.nii.gz", "disease_mask": "seg_rxg/train_214_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Linear subsegmental atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_269/train_269_a/train_269_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_269_a_2.nii.gz", "disease_mask": "seg_rxg/train_269_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Two indeterminate subcentimeter nodules in the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_269/train_269_a/train_269_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_269_a_2.nii.gz", "disease_mask": "seg_rxg/train_269_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule measuring 1.5 mm in the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_12/train_12_a/train_12_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12_a_2.nii.gz", "disease_mask": "seg_rxg/train_12_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_12/train_12_a/train_12_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12_a_2.nii.gz", "disease_mask": "seg_rxg/train_12_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Fibroatelectatic scarring in the right middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12/train_12_a/train_12_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12_a_2.nii.gz", "disease_mask": "seg_rxg/train_12_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Fibroatelectatic scarring in the lingular segment of the left upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_12/train_12_a/train_12_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12_a_2.nii.gz", "disease_mask": "seg_rxg/train_12_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Fibroatelectatic scarring in the basal segments of both lower lobes", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_389/train_389_a/train_389_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_389_a_2.nii.gz", "disease_mask": "seg_rxg/train_389_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_389/train_389_a/train_389_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_389_a_2.nii.gz", "disease_mask": "seg_rxg/train_389_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_389/train_389_a/train_389_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_389_a_2.nii.gz", "disease_mask": "seg_rxg/train_389_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_389/train_389_a/train_389_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_389_a_2.nii.gz", "disease_mask": "seg_rxg/train_389_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pleural effusion or thickening", "disease_findings": "Small right pleural effusion", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_119/train_119_a/train_119_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_119_a_2.nii.gz", "disease_mask": "seg_rxg/train_119_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Centrally located mass in the right upper lobe measuring 6 cm in diameter adjacent to the mediastinum and enveloping the upper lobe bronchi", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_119/train_119_a/train_119_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_119_a_2.nii.gz", "disease_mask": "seg_rxg/train_119_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Thin-walled air cyst in the lateral segment of the right middle lobe, measuring 3.8 x 2.4 cm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_119/train_119_a/train_119_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_119_a_2.nii.gz", "disease_mask": "seg_rxg/train_119_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Three closely spaced pulmonary parenchymal nodules in the anterior segment of the left upper lobe, each measuring 4 mm, decreased in size on follow-up", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_119/train_119_a/train_119_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_119_a_2.nii.gz", "disease_mask": "seg_rxg/train_119_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule in the anterior aspect of the right lung measuring 7 mm in diameter, slightly more prominent on follow-up", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_344/train_344_d/train_344_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_344_d_2.nii.gz", "disease_mask": "seg_rxg/train_344_d_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Bilateral paraseptal emphysematous changes", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_344/train_344_d/train_344_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_344_d_2.nii.gz", "disease_mask": "seg_rxg/train_344_d_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Areas of minimal ground-glass opacity in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_371/train_371_a/train_371_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_371_a_2.nii.gz", "disease_mask": "seg_rxg/train_371_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple indeterminate parenchymal nodules up to 4.4 mm in diameter within both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_371/train_371_a/train_371_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_371_a_2.nii.gz", "disease_mask": "seg_rxg/train_371_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest nodule located in the middle lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_78/train_78_a/train_78_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_78_a_2.nii.gz", "disease_mask": "seg_rxg/train_78_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Paraseptal emphysematous changes in the upper lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_78/train_78_a/train_78_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_78_a_2.nii.gz", "disease_mask": "seg_rxg/train_78_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Fibrotic bands and scarring in the upper lobes of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6/train_6_a/train_6_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6_a_2.nii.gz", "disease_mask": "seg_rxg/train_6_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Diffuse interstitial thickening throughout all lung segments", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_6/train_6_a/train_6_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6_a_2.nii.gz", "disease_mask": "seg_rxg/train_6_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities throughout all lung segments", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_81/train_81_a/train_81_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_81_a_2.nii.gz", "disease_mask": "seg_rxg/train_81_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with air bronchograms in the inferior lingular segment of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_81/train_81_a/train_81_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_81_a_2.nii.gz", "disease_mask": "seg_rxg/train_81_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities with centriacinar nodules in the left lung adjacent parenchyma", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_81/train_81_a/train_81_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_81_a_2.nii.gz", "disease_mask": "seg_rxg/train_81_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with air bronchograms in the right lung middle lobe's lateral and medial segments", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_81/train_81_a/train_81_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_81_a_2.nii.gz", "disease_mask": "seg_rxg/train_81_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with ground glass densities and air bronchograms extending to the pleura in the posterobasal segment of the left lung lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_81/train_81_a/train_81_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_81_a_2.nii.gz", "disease_mask": "seg_rxg/train_81_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with ground glass densities and air bronchograms in the posterior anteromedial segment of the left lung lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_81/train_81_a/train_81_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_81_a_2.nii.gz", "disease_mask": "seg_rxg/train_81_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with ground glass densities and air bronchograms in the anterior and posterior segments of the right lung lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_81/train_81_a/train_81_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_81_a_2.nii.gz", "disease_mask": "seg_rxg/train_81_a_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple nonspecific nodules in the parenchyma of both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_588/train_588_a/train_588_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_588_a_2.nii.gz", "disease_mask": "seg_rxg/train_588_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Bilateral nodular subpleural ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_619/train_619_a/train_619_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_619_a_2.nii.gz", "disease_mask": "seg_rxg/train_619_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Bilateral patchy ground glass opacities in the lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_619/train_619_a/train_619_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_619_a_2.nii.gz", "disease_mask": "seg_rxg/train_619_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of consolidation in the lungs, predominantly in the lower lobes", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_634/train_634_a/train_634_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_634_a_2.nii.gz", "disease_mask": "seg_rxg/train_634_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Bilateral diffuse patchy ground glass opacities", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_669/train_669_a/train_669_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_669_a_2.nii.gz", "disease_mask": "seg_rxg/train_669_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_456/train_456_b/train_456_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_456_b_2.nii.gz", "disease_mask": "seg_rxg/train_456_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Bilateral peripheral subpleural nodular ground glass opacities within the lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_570/train_570_a/train_570_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_570_a_2.nii.gz", "disease_mask": "seg_rxg/train_570_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Minimal peribronchial thickening bilaterally", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_542/train_542_a/train_542_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_542_a_2.nii.gz", "disease_mask": "seg_rxg/train_542_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular-patchy consolidations forming a crazy paving pattern in the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_542/train_542_a/train_542_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_542_a_2.nii.gz", "disease_mask": "seg_rxg/train_542_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular-patchy consolidations forming a crazy paving pattern in the upper lobe of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_542/train_542_a/train_542_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_542_a_2.nii.gz", "disease_mask": "seg_rxg/train_542_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular-patchy consolidations forming a crazy paving pattern in the lower lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_542/train_542_a/train_542_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_542_a_2.nii.gz", "disease_mask": "seg_rxg/train_542_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Widespread subsegmental atelectasis in the lower lobes", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_784/train_784_f/train_784_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_f_2.nii.gz", "disease_mask": "seg_rxg/train_784_f_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Centrilobular nodular opacities with tree-in-bud patterns in both lungs, predominantly on the right", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_784/train_784_f/train_784_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_f_2.nii.gz", "disease_mask": "seg_rxg/train_784_f_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules in the right lung apex, measuring 6 mm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_784/train_784_f/train_784_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_f_2.nii.gz", "disease_mask": "seg_rxg/train_784_f_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules in the left lung upper lobe's inferior lingula, measuring 6 mm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_784/train_784_f/train_784_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_f_2.nii.gz", "disease_mask": "seg_rxg/train_784_f_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Bronchiectasis", "disease_findings": "Bronchiectasis at the aforementioned levels", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_784/train_784_f/train_784_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_f_2.nii.gz", "disease_mask": "seg_rxg/train_784_f_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Emphysema", "disease_findings": "Bilateral apical emphysematous changes", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_784/train_784_f/train_784_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_f_2.nii.gz", "disease_mask": "seg_rxg/train_784_f_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Linear", "disease_findings": "Parenchymal scarring in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_784/train_784_f/train_784_f_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_f_2.nii.gz", "disease_mask": "seg_rxg/train_784_f_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple stable subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_510/train_510_a/train_510_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_510_a_2.nii.gz", "disease_mask": "seg_rxg/train_510_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Bilateral pulmonary apices, superior segment of the right lower lobe, posterobasal segment, medial and lateral segments, and inferior segments of the left lower lobe and lingula with structural distortion, volume loss, bronchiectasis, and increased interlobular septal thickness", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_492/train_492_a/train_492_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_492_a_2.nii.gz", "disease_mask": "seg_rxg/train_492_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Localized area of ground-glass opacity in the peripheral region of the mediobasal segment of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_622/train_622_a/train_622_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_622_a_2.nii.gz", "disease_mask": "seg_rxg/train_622_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Fibrotic parenchymal changes in the left lung upper lobe inferior lingular segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_622/train_622_a/train_622_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_622_a_2.nii.gz", "disease_mask": "seg_rxg/train_622_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Fibrotic parenchymal changes in the right lung middle lobe medial segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_622/train_622_a/train_622_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_622_a_2.nii.gz", "disease_mask": "seg_rxg/train_622_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Fibrotic parenchymal changes in the right lung upper lobe anterior segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_622/train_622_a/train_622_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_622_a_2.nii.gz", "disease_mask": "seg_rxg/train_622_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule in the right lung middle lobe medial segment measuring 9 mm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_622/train_622_a/train_622_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_622_a_2.nii.gz", "disease_mask": "seg_rxg/train_622_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule in the right lung lower lobe laterobasal segment measuring 7 mm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_622/train_622_a/train_622_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_622_a_2.nii.gz", "disease_mask": "seg_rxg/train_622_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Additional subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_622/train_622_a/train_622_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_622_a_2.nii.gz", "disease_mask": "seg_rxg/train_622_a_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Micronodules", "disease_findings": "Calcified nodules in both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_784/train_784_d/train_784_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_d_2.nii.gz", "disease_mask": "seg_rxg/train_784_d_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Centrilobular nodular densities with a tree-in-bud pattern in the lateral aspects of both upper lobes", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_784/train_784_d/train_784_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_d_2.nii.gz", "disease_mask": "seg_rxg/train_784_d_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Centrilobular nodular densities with a tree-in-bud pattern in the superior segment of the right lower lobe", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_784/train_784_d/train_784_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_d_2.nii.gz", "disease_mask": "seg_rxg/train_784_d_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Micronodules", "disease_findings": "Centrilobular nodular densities with a tree-in-bud pattern in the posterobasal segments of both lower lobes", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_784/train_784_d/train_784_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_d_2.nii.gz", "disease_mask": "seg_rxg/train_784_d_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_784/train_784_d/train_784_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_d_2.nii.gz", "disease_mask": "seg_rxg/train_784_d_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Fibrosis in the bilateral apical segments", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_784/train_784_d/train_784_d_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_d_2.nii.gz", "disease_mask": "seg_rxg/train_784_d_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_739/train_739_a/train_739_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_739_a_2.nii.gz", "disease_mask": "seg_rxg/train_739_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple subcentimeter nodules in both lungs, predominantly subpleural", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_739/train_739_a/train_739_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_739_a_2.nii.gz", "disease_mask": "seg_rxg/train_739_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest nodule measuring 5 mm at anterior lateral segment junction of right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_473/train_473_a/train_473_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_473_a_2.nii.gz", "disease_mask": "seg_rxg/train_473_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes bilaterally in the lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_473/train_473_a/train_473_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_473_a_2.nii.gz", "disease_mask": "seg_rxg/train_473_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the lower lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_825/train_825_a/train_825_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_825_a_2.nii.gz", "disease_mask": "seg_rxg/train_825_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Fibrotic changes at the apices of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_825/train_825_a/train_825_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_825_a_2.nii.gz", "disease_mask": "seg_rxg/train_825_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Well-circumscribed nodule measuring 8.9x6.7 mm in the posterior segment of the right upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_600/train_600_a/train_600_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_600_a_2.nii.gz", "disease_mask": "seg_rxg/train_600_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Bilateral lower lung lobes and the right upper lobe nodules with surrounding ground glass opacities in peribronchovascular and subpleural regions", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_428/train_428_a/train_428_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_428_a_2.nii.gz", "disease_mask": "seg_rxg/train_428_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_428/train_428_a/train_428_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_428_a_2.nii.gz", "disease_mask": "seg_rxg/train_428_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_519/train_519_b/train_519_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_519_b_2.nii.gz", "disease_mask": "seg_rxg/train_519_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Bilateral upper and lower lobes ground glass opacities", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_519/train_519_b/train_519_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_519_b_2.nii.gz", "disease_mask": "seg_rxg/train_519_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities in the right middle lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_519/train_519_b/train_519_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_519_b_2.nii.gz", "disease_mask": "seg_rxg/train_519_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Septal thickening", "disease_findings": "Accompanying interlobular septal thickening", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_519/train_519_b/train_519_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_519_b_2.nii.gz", "disease_mask": "seg_rxg/train_519_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Linear", "disease_findings": "Linear densities parallel to the pleura", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_519/train_519_b/train_519_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_519_b_2.nii.gz", "disease_mask": "seg_rxg/train_519_b_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Linear densities in subpleural areas", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_784/train_784_c/train_784_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_c_2.nii.gz", "disease_mask": "seg_rxg/train_784_c_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_784/train_784_c/train_784_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_c_2.nii.gz", "disease_mask": "seg_rxg/train_784_c_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Scarring at both lung apices", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_784/train_784_c/train_784_c_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_784_c_2.nii.gz", "disease_mask": "seg_rxg/train_784_c_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Indeterminate subcentimeter nodules within both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1042/train_1042_a/train_1042_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1042_a_2.nii.gz", "disease_mask": "seg_rxg/train_1042_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular consolidations in the anterior segment of the right upper lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1042/train_1042_a/train_1042_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1042_a_2.nii.gz", "disease_mask": "seg_rxg/train_1042_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular consolidations in the superior segment of the right lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_877/train_877_a/train_877_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_877_a_2.nii.gz", "disease_mask": "seg_rxg/train_877_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Thickening of the pleura with calcified foci", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_877/train_877_a/train_877_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_877_a_2.nii.gz", "disease_mask": "seg_rxg/train_877_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pleural effusion or thickening", "disease_findings": "Nodular thickening at the major fissure, measuring 2.5 cm at its thickest point", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_877/train_877_a/train_877_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_877_a_2.nii.gz", "disease_mask": "seg_rxg/train_877_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Fibrotic changes in the inferior lingular segment and superior segment of the left lower lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_877/train_877_a/train_877_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_877_a_2.nii.gz", "disease_mask": "seg_rxg/train_877_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Emphysema", "disease_findings": "Paraseptal emphysematous changes at the apices of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_877/train_877_a/train_877_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_877_a_2.nii.gz", "disease_mask": "seg_rxg/train_877_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Micronodules", "disease_findings": "Multiple calcified and subcentimeter nodules in both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_826/train_826_b/train_826_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_826_b_2.nii.gz", "disease_mask": "seg_rxg/train_826_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral and central ground glass opacities in the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_826/train_826_b/train_826_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_826_b_2.nii.gz", "disease_mask": "seg_rxg/train_826_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodules with a tree-in-bud pattern in the right lung", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_826/train_826_b/train_826_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_826_b_2.nii.gz", "disease_mask": "seg_rxg/train_826_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of consolidation in the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_826/train_826_b/train_826_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_826_b_2.nii.gz", "disease_mask": "seg_rxg/train_826_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "New findings in the anteromediobasal segment of the right lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1227/train_1227_b/train_1227_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1227_b_2.nii.gz", "disease_mask": "seg_rxg/train_1227_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple indeterminate subcentimeter pulmonary nodules in the anterior segment of the right upper lobe and both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1227/train_1227_b/train_1227_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1227_b_2.nii.gz", "disease_mask": "seg_rxg/train_1227_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Mild emphysematous changes bilaterally", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_1227/train_1227_b/train_1227_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1227_b_2.nii.gz", "disease_mask": "seg_rxg/train_1227_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Stable parenchymal fibrosis and post-scar bronchiectatic changes causing structural distortion and volume loss in the upper lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1227/train_1227_b/train_1227_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1227_b_2.nii.gz", "disease_mask": "seg_rxg/train_1227_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Bronchiectasis", "disease_findings": "Bronchiectatic changes bilaterally", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_835/train_835_a/train_835_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_835_a_2.nii.gz", "disease_mask": "seg_rxg/train_835_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs with blebs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_835/train_835_a/train_835_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_835_a_2.nii.gz", "disease_mask": "seg_rxg/train_835_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Paraseptal emphysematous changes more pronounced at the apexes", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_835/train_835_a/train_835_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_835_a_2.nii.gz", "disease_mask": "seg_rxg/train_835_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Multiple pulmonary nodules bilaterally, largest 5 mm in diameter", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1013/train_1013_a/train_1013_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1013_a_2.nii.gz", "disease_mask": "seg_rxg/train_1013_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Diffuse bilateral ground glass opacification within the lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1013/train_1013_a/train_1013_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1013_a_2.nii.gz", "disease_mask": "seg_rxg/train_1013_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atypical pneumonic infiltrates in the basal segments of the lower lobes", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1013/train_1013_a/train_1013_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1013_a_2.nii.gz", "disease_mask": "seg_rxg/train_1013_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Accompanying linear atelectasis in the basal segments of the lower lobes", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1013/train_1013_a/train_1013_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1013_a_2.nii.gz", "disease_mask": "seg_rxg/train_1013_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Widespread lung parenchymal involvement presenting as ground glass densities", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_848/train_848_i/train_848_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_848_i_2.nii.gz", "disease_mask": "seg_rxg/train_848_i_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Bronchial wall thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_848/train_848_i/train_848_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_848_i_2.nii.gz", "disease_mask": "seg_rxg/train_848_i_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Pulmonary fibrosis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1124/train_1124_a/train_1124_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1124_a_2.nii.gz", "disease_mask": "seg_rxg/train_1124_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Marked centrilobular emphysema in the upper lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_1124/train_1124_a/train_1124_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1124_a_2.nii.gz", "disease_mask": "seg_rxg/train_1124_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear fibrotic changes in the posterior segment of the right upper lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1124/train_1124_a/train_1124_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1124_a_2.nii.gz", "disease_mask": "seg_rxg/train_1124_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectasis in the lingula of the left upper lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1124/train_1124_a/train_1124_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1124_a_2.nii.gz", "disease_mask": "seg_rxg/train_1124_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Stable 3 mm subpleural nodule in the lingula of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1157/train_1157_e/train_1157_e_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1157_e_2.nii.gz", "disease_mask": "seg_rxg/train_1157_e_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Bilateral pulmonary lesions consistent with metastases", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1157/train_1157_e/train_1157_e_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1157_e_2.nii.gz", "disease_mask": "seg_rxg/train_1157_e_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Cavitary lesion in the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1157/train_1157_e/train_1157_e_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1157_e_2.nii.gz", "disease_mask": "seg_rxg/train_1157_e_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Cavitary lesion with central cavitation", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1157/train_1157_e/train_1157_e_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1157_e_2.nii.gz", "disease_mask": "seg_rxg/train_1157_e_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Cavitary lesion with a maximum diameter of approximately 40 mm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1157/train_1157_e/train_1157_e_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1157_e_2.nii.gz", "disease_mask": "seg_rxg/train_1157_e_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subpleural lesion with a maximum diameter of 23 mm", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1157/train_1157_e/train_1157_e_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1157_e_2.nii.gz", "disease_mask": "seg_rxg/train_1157_e_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subpleural lesion increased from 8 mm on previous imaging", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_856/train_856_a/train_856_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_856_a_2.nii.gz", "disease_mask": "seg_rxg/train_856_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchial wall thickening", "disease_findings": "Peribronchial thickening in the lower lobe of the right lung", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_856/train_856_a/train_856_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_856_a_2.nii.gz", "disease_mask": "seg_rxg/train_856_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Patchy interlobular septal and interstitial thickenings in the right lung", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_856/train_856_a/train_856_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_856_a_2.nii.gz", "disease_mask": "seg_rxg/train_856_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Mild architectural distortion and volume loss in the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_856/train_856_a/train_856_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_856_a_2.nii.gz", "disease_mask": "seg_rxg/train_856_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Absence of previously noted primary mass in the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_856/train_856_a/train_856_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_856_a_2.nii.gz", "disease_mask": "seg_rxg/train_856_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Linear", "disease_findings": "Post-inflammatory scarring in the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_856/train_856_a/train_856_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_856_a_2.nii.gz", "disease_mask": "seg_rxg/train_856_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Bronchiectasis", "disease_findings": "Mild bronchiectasis and peribronchial thickening in the central part of the left lung", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_856/train_856_a/train_856_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_856_a_2.nii.gz", "disease_mask": "seg_rxg/train_856_a_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_856/train_856_a/train_856_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_856_a_2.nii.gz", "disease_mask": "seg_rxg/train_856_a_2.nii.gz", "disease_mask_channel": "7", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Stable 14 mm nodule in the lateral segment of the right middle lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1147/train_1147_a/train_1147_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1147_a_2.nii.gz", "disease_mask": "seg_rxg/train_1147_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Smooth interlobular and interstitial septal thickening", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_1141/train_1141_a/train_1141_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1141_a_2.nii.gz", "disease_mask": "seg_rxg/train_1141_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Fibrotic changes in the minor fissure of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1213/train_1213_a/train_1213_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1213_a_2.nii.gz", "disease_mask": "seg_rxg/train_1213_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with air bronchograms in the right lung, predominantly within the middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1213/train_1213_a/train_1213_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1213_a_2.nii.gz", "disease_mask": "seg_rxg/train_1213_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Primary mass in the lower lobe of the right lung obscured by consolidation", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1213/train_1213_a/train_1213_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1213_a_2.nii.gz", "disease_mask": "seg_rxg/train_1213_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Soft tissue densities in the upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1213/train_1213_a/train_1213_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1213_a_2.nii.gz", "disease_mask": "seg_rxg/train_1213_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Bronchial wall thickening", "disease_findings": "Peribronchial thickening in the left lung", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_1213/train_1213_a/train_1213_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1213_a_2.nii.gz", "disease_mask": "seg_rxg/train_1213_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Micronodules", "disease_findings": "Scattered centrilobular nodules in the left lung, predominance in the lower lobe", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_1123/train_1123_a/train_1123_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1123_a_2.nii.gz", "disease_mask": "seg_rxg/train_1123_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Bilateral linear atelectasis in the lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_1123/train_1123_a/train_1123_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1123_a_2.nii.gz", "disease_mask": "seg_rxg/train_1123_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Several subcentimeter non-specific nodules in the lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1123/train_1123_a/train_1123_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1123_a_2.nii.gz", "disease_mask": "seg_rxg/train_1123_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules with calcifications measuring less than 3 mm in short axis", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_900/train_900_a/train_900_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_900_a_2.nii.gz", "disease_mask": "seg_rxg/train_900_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Bilateral lung parenchyma with a predominant interstitial pattern", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_900/train_900_a/train_900_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_900_a_2.nii.gz", "disease_mask": "seg_rxg/train_900_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Peripheral distribution of interstitial pattern in bilateral lung parenchyma", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_900/train_900_a/train_900_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_900_a_2.nii.gz", "disease_mask": "seg_rxg/train_900_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Honeycombing", "disease_findings": "Honeycombing indicative of interstitial lung disease", "disease_class": 155, "disease_label_text": "Honeycombing"} +{"volume_path": "dataset/train_fixed/train_850/train_850_a/train_850_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_850_a_2.nii.gz", "disease_mask": "seg_rxg/train_850_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes within both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_873/train_873_b/train_873_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_873_b_2.nii.gz", "disease_mask": "seg_rxg/train_873_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Area of increased density with air bronchograms in the left upper lobe's inferior lingular segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_873/train_873_b/train_873_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_873_b_2.nii.gz", "disease_mask": "seg_rxg/train_873_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Smooth interlobular septal thickening in the right lung's middle and lower lobes", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_1105/train_1105_a/train_1105_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1105_a_2.nii.gz", "disease_mask": "seg_rxg/train_1105_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Diffuse mild bronchiectasis within central bronchial structures bilaterally", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1105/train_1105_a/train_1105_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1105_a_2.nii.gz", "disease_mask": "seg_rxg/train_1105_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchial wall thickening", "disease_findings": "Peribronchial thickening within central bronchial structures bilaterally", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_1105/train_1105_a/train_1105_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1105_a_2.nii.gz", "disease_mask": "seg_rxg/train_1105_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with ground-glass opacities in the right lung lower lobe posterobasal segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1105/train_1105_a/train_1105_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1105_a_2.nii.gz", "disease_mask": "seg_rxg/train_1105_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with ground-glass opacities in the right lung middle lobe lateral segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1105/train_1105_a/train_1105_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1105_a_2.nii.gz", "disease_mask": "seg_rxg/train_1105_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Atelectasis, consolidation", "disease_findings": "Peripheral subpleural consolidation in the right lung lower lobe posterobasal segment measuring approximately 1.5x1 cm with indistinct margins", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_904/train_904_a/train_904_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_904_a_2.nii.gz", "disease_mask": "seg_rxg/train_904_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Bilateral multilobar, peripherally located crazy paving patterns", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_904/train_904_a/train_904_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_904_a_2.nii.gz", "disease_mask": "seg_rxg/train_904_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Areas of nodular ground glass density", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_914/train_914_a/train_914_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_914_a_2.nii.gz", "disease_mask": "seg_rxg/train_914_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5231/train_5231_a/train_5231_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5231_a_2.nii.gz", "disease_mask": "seg_rxg/train_5231_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Peripheral and central consolidations in the upper and lower lobes of both lungs and the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5231/train_5231_a/train_5231_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5231_a_2.nii.gz", "disease_mask": "seg_rxg/train_5231_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the upper and lower lobes of both lungs and the middle lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5231/train_5231_a/train_5231_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5231_a_2.nii.gz", "disease_mask": "seg_rxg/train_5231_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest lesions located in the posterobasal segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7758/train_7758_b/train_7758_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7758_b_2.nii.gz", "disease_mask": "seg_rxg/train_7758_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few subcentimeter nodules, some of which are calcified and nonspecific, in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7693/train_7693_a/train_7693_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7693_a_2.nii.gz", "disease_mask": "seg_rxg/train_7693_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1653/train_1653_a/train_1653_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1653_a_2.nii.gz", "disease_mask": "seg_rxg/train_1653_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral localized ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5483/train_5483_a/train_5483_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5483_a_2.nii.gz", "disease_mask": "seg_rxg/train_5483_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Occasional areas of atelectasis in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5483/train_5483_a/train_5483_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5483_a_2.nii.gz", "disease_mask": "seg_rxg/train_5483_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass appearance in the peripheral area of the anterior segment of the right upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5483/train_5483_a/train_5483_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5483_a_2.nii.gz", "disease_mask": "seg_rxg/train_5483_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Septal thickening", "disease_findings": "Interlobular septal thickening in the peripheral area of the anterior segment of the right upper lobe", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_11165/train_11165_a/train_11165_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11165_a_2.nii.gz", "disease_mask": "seg_rxg/train_11165_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Mild paraseptal emphysematous changes at the left apical level", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_7294/train_7294_a/train_7294_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7294_a_2.nii.gz", "disease_mask": "seg_rxg/train_7294_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Central-peripheral localized nodular ground glass opacities with faint borders in the basal segment of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7294/train_7294_a/train_7294_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7294_a_2.nii.gz", "disease_mask": "seg_rxg/train_7294_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific calcific nodule in the posterior subsegment of the left upper lobe apicoposterior segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_428/train_428_a/train_428_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_428_a_2.nii.gz", "disease_mask": "seg_rxg/train_428_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_428/train_428_a/train_428_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_428_a_2.nii.gz", "disease_mask": "seg_rxg/train_428_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2977/train_2977_a/train_2977_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2977_a_2.nii.gz", "disease_mask": "seg_rxg/train_2977_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Widespread subpleural ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2977/train_2977_a/train_2977_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2977_a_2.nii.gz", "disease_mask": "seg_rxg/train_2977_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidations in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_2977/train_2977_a/train_2977_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2977_a_2.nii.gz", "disease_mask": "seg_rxg/train_2977_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchial wall thickening", "disease_findings": "Focal bronchial enlargement within areas of consolidation", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_1441/train_1441_a/train_1441_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1441_a_2.nii.gz", "disease_mask": "seg_rxg/train_1441_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "4 mm non-contouring opacity with a minor fissure in the anterior segment of the upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1441/train_1441_a/train_1441_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1441_a_2.nii.gz", "disease_mask": "seg_rxg/train_1441_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectasis in the inferior segment of the left lung lingula", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6954/train_6954_a/train_6954_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6954_a_2.nii.gz", "disease_mask": "seg_rxg/train_6954_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "10 mm nodular ground glass opacity in the posterobasal segment of the left lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6954/train_6954_a/train_6954_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6954_a_2.nii.gz", "disease_mask": "seg_rxg/train_6954_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Fibrotic changes in the apices of both upper lobes", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_6954/train_6954_a/train_6954_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6954_a_2.nii.gz", "disease_mask": "seg_rxg/train_6954_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Fibrotic band in the right middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5460/train_5460_b/train_5460_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5460_b_2.nii.gz", "disease_mask": "seg_rxg/train_5460_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral and centrally located ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5460/train_5460_b/train_5460_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5460_b_2.nii.gz", "disease_mask": "seg_rxg/train_5460_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Linear densities accompanying ground glass opacities", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_5460/train_5460_b/train_5460_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5460_b_2.nii.gz", "disease_mask": "seg_rxg/train_5460_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Occasional consolidations accompanying ground glass opacities", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} diff --git a/Abnormal-CT-Generation-MultiDisease/inference_train_multi.py b/Abnormal-CT-Generation-MultiDisease/inference_train_multi.py new file mode 100644 index 0000000000000000000000000000000000000000..cbaf22086380139f8f3adb0e41e42382b64aa9d6 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/inference_train_multi.py @@ -0,0 +1,385 @@ +import glob +import nibabel as nib +import numpy as np +import os +import torch + +from einops import rearrange +from omegaconf import OmegaConf +from torch.utils.data import DataLoader +from tqdm import tqdm + +from ldm.util import instantiate_from_config +import argparse + + +import os +import re + +def fix_filename(filepath): + """ + 修复包含特殊字符的文件路径,将斜杠等特殊字符替换为下划线 + + Args: + filepath: 原始文件路径 + + Returns: + 修复后的文件路径 + """ + # 分离目录和文件名 + directory = os.path.dirname(filepath) + filename = os.path.basename(filepath) + + # 替换文件名中的特殊字符为下划线 + # 包括:斜杠、反斜杠、冒号、星号、问号、引号、尖括号、竖线 + clean_filename = re.sub(r'[/\\:*?"<>|]', '_', filename) + + # 处理连续的下划线,替换为单个下划线 + clean_filename = re.sub(r'_+', '_', clean_filename) + + # 去除首尾的下划线 + clean_filename = clean_filename.strip('_') + + # 组合新路径 + if directory: + return os.path.join(directory, clean_filename) + else: + return clean_filename + + + +def compute_orientation(init_axcodes, final_axcodes): + """ + A thin wrapper around ``nib.orientations.ornt_transform`` + + :param init_axcodes: Initial orientation codes + :param final_axcodes: Target orientation codes + :return: orientations array, start_ornt, end_ornt + """ + ornt_init = nib.orientations.axcodes2ornt(init_axcodes) + ornt_fin = nib.orientations.axcodes2ornt(final_axcodes) + + ornt_transf = nib.orientations.ornt_transform(ornt_init, ornt_fin) + + return ornt_transf, ornt_init, ornt_fin + +def do_reorientation(data_array, init_axcodes, final_axcodes): + """ + source: https://niftynet.readthedocs.io/en/dev/_modules/niftynet/io/misc_io.html#do_reorientation + Performs the reorientation (changing order of axes) + + :param data_array: 3D Array to reorient + :param init_axcodes: Initial orientation + :param final_axcodes: Target orientation + :return data_reoriented: New data array in its reoriented form + """ + ornt_transf, ornt_init, ornt_fin = compute_orientation(init_axcodes, final_axcodes) + if np.array_equal(ornt_init, ornt_fin): + return data_array + + return nib.orientations.apply_orientation(data_array, ornt_transf) + +def compute_fusion_boundaries(upper_iters, latent_length, decode_ratio=4, blend_width=16): + """ + Compute fusion boundaries based on result segments + + :param upper_iters: Number of iterations in the generation loop + :param latent_length: Length of latent dimension (17) + :param decode_ratio: Ratio between decoded and latent dimensions (4) + :param blend_width: Width of blending region on each side (16 slices) + :return: List of boundary dictionaries for fusion + """ + boundaries = [] + + for i in range(1, upper_iters): + # Calculate boundary position in latent space + latent_boundary = i * latent_length + + # Convert to decoded space (multiply by decode_ratio) + decoded_boundary = latent_boundary * decode_ratio + 1 + + # Define blending region: boundary ± blend_width + blend_start = max(0, decoded_boundary - blend_width) + blend_end = decoded_boundary + blend_width + + boundaries.append({ + 'boundary': decoded_boundary, + 'blend_start': blend_start, + 'blend_end': blend_end + }) + + return boundaries + + +def alternating_volume_fusion_with_boundaries(vol1, vol2, boundaries, offset=32, replace_width=16): + """ + Fuse two volumes by replacing regions around boundaries + + :param vol1: First volume with shape (x, y, z) - used as base + :param vol2: Second volume with shape (x, y, z-offset), offset by 32 slices + :param boundaries: List of boundary dictionaries from compute_fusion_boundaries + :param offset: Offset between vol1 and vol2 (default 32) + :param replace_width: Width on each side of boundary to replace (default 16) + :return: Fused volume with vol2 replacing vol1 at boundary regions + """ + x, y, total_slices = vol1.shape + fused = vol1.copy() # Start with vol1 as base + + # Sort boundaries by position + sorted_boundaries = sorted(boundaries, key=lambda b: b['boundary']) + + print(f"Fusion starting with vol1 as base") + + for i, boundary_info in enumerate(sorted_boundaries): + boundary = boundary_info['boundary'] + + # Define replacement region: boundary ± replace_width + replace_start = max(0, boundary - replace_width) + replace_end = min(boundary + replace_width, total_slices) + + # Map to vol2 coordinates + vol2_start = replace_start - offset + vol2_end = replace_end - offset + + # Replace this region with vol2 if valid + if vol2_start >= 0 and vol2_end <= vol2.shape[2]: + fused[:, :, replace_start:replace_end] = vol2[:, :, vol2_start:vol2_end] + print(f" Boundary {i+1}: z={boundary}, replaced z[{replace_start}:{replace_end}] with vol2[{vol2_start}:{vol2_end}]") + else: + print(f" Boundary {i+1}: z={boundary}, skipped (vol2 out of range)") + + print(f"Fusion complete with {len(sorted_boundaries)} boundaries") + + return fused + +parser = argparse.ArgumentParser() +parser.add_argument( + "-t", + "--time_steps", + type=int, + default=20, +) +args = parser.parse_args() +ddim_steps=args.time_steps + +logdir = 'logs/full_ct_3d_with_body_mask2' +ckpt = os.path.join(logdir, "checkpoints", "epoch=000095.ckpt") + +configs_file = "configs/latent-diffusion/infer_train_multi.yaml" +configs = OmegaConf.load(configs_file) +model = instantiate_from_config(configs.model) +model.init_from_ckpt(ckpt) +model.eval() + +device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") +print("Using device", device) +model = model.to(device) + +config = OmegaConf.load('./configs/latent-diffusion/infer_train_multi.yaml') +data = instantiate_from_config(config.data) +data.prepare_data() +data.setup() + +save_path = f'inference_valid' + +if not os.path.exists(save_path): + os.makedirs(save_path) + +val_dataset = data.datasets['validation'] +batch_size = 1 +valloader = DataLoader(val_dataset, batch_size=batch_size, shuffle=False, num_workers=2, pin_memory=True) +val_num = len(val_dataset) +save_gt = True + +for idx, data in tqdm(enumerate(valloader)): + + if idx >= val_num: + break + + name=data['name'][0].split('.')[0] + volume_data = data['volume_data'] + volume_seg = data['volume_seg'] + # disease_mask_channel=data['disease_mask_channel'] + # disease_label=data['disease_label'][0] + # breakpoint() + + data_path = os.path.join(save_path, str(f'{name}.nii.gz').replace('/', '_')) + if os.path.exists(data_path): + print(f"File already exists, skipping: {data_path}") + continue + + print(volume_data.shape) + window_length = 65 + latent_lenght=17 + slice_num = volume_data.shape[2] + + upper_iters = slice_num // window_length + 1 if slice_num % window_length != 0 else slice_num // window_length + result = torch.zeros((batch_size, upper_iters*latent_lenght, 16, 64, 64)).cuda() + print('upper_iters', upper_iters) + + for i in range(upper_iters): + print('i', i) + input_data={} + if i == upper_iters-1: + input_data['name'] = data['name'] + input_data['volume_data'] = data['volume_data'][:, :, -window_length:].to(device) + input_data['volume_seg'] = data['volume_seg'][:, :, -window_length:].to(device) + input_data['input_text'] = data['input_text'] + else: + input_data['volume_data'] = data['volume_data'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['volume_seg'] = data['volume_seg'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['input_text'] = data['input_text'] + + with torch.no_grad(): + _, c = model.get_input(input_data, model.first_stage_key) + + if i == 0: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps) + else: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps, previous=x_minus1) + + samples_i = rearrange(samples_i, '(b z) c h w -> b z c h w', z=latent_lenght) + + if i == upper_iters-1: + result[:, -latent_lenght:] = samples_i + else: + result[:, i*latent_lenght:(i+1)*latent_lenght] = samples_i + x_minus1 = samples_i[:, -1:, ...] # Take last slice for next iteration + + result = result.permute(0,2,1,3,4) + x_result = torch.zeros((3,slice_num,512,512)) + dec_unit = 65 + dec_latent_unit=17 + num_dec_iter = slice_num // dec_unit + 1 if slice_num % dec_unit != 0 else slice_num // dec_unit + + for i in range(num_dec_iter): + if i == num_dec_iter - 1: + remaining_slices = slice_num % dec_unit if slice_num % dec_unit != 0 else dec_unit + x_result[:, -remaining_slices:] = model.decode_first_stage(result[:, :, -latent_lenght:])[0][:, -remaining_slices:] + else: + x_result[:, i*dec_unit:(i+1)*dec_unit] = model.decode_first_stage(result[:, :, i*latent_lenght:(i+1)*latent_lenght])[0] + + x_result = x_result*2 + x_result[x_result>1.0] = 1.0 + x_result[x_result<-1.0] = -1.0 + x_result = (x_result+1)/2 + x_result_ = x_result.mean(axis=0).detach().cpu().numpy() + x_result = x_result_.transpose(1,2,0) + + # Process second volume with offset + data2={} + data2['volume_data'] = data['volume_data'][:, :, 32:] + data2['volume_seg'] = data['volume_seg'][:, :, 32:] + data2['name'] = data['name'] + data2['input_text'] = data['input_text'] + + volume_data = data2['volume_data'] + volume_seg = data2['volume_seg'] + + window_length = 65 + latent_lenght=17 + slice_num2 = volume_data.shape[2] + + upper_iters2 = slice_num2 // window_length + 1 if slice_num2 % window_length != 0 else slice_num2 // window_length + result2 = torch.zeros((batch_size, upper_iters2*latent_lenght, 16, 64, 64)).cuda() + print('upper_iters2', upper_iters2) + + for i in range(upper_iters2): + print('i', i) + input_data={} + if i == upper_iters2-1: + input_data['name'] = data2['name'] + input_data['volume_data'] = data2['volume_data'][:, :, -window_length:].to(device) + input_data['volume_seg'] = data2['volume_seg'][:, :, -window_length:].to(device) + input_data['input_text'] = data2['input_text'] + else: + input_data['volume_data'] = data2['volume_data'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['volume_seg'] = data2['volume_seg'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['input_text'] = data2['input_text'] + + with torch.no_grad(): + _, c = model.get_input(input_data, model.first_stage_key) + + if i == 0: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps) + else: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps, previous=x_minus1) + + samples_i = rearrange(samples_i, '(b z) c h w -> b z c h w', z=latent_lenght) + + if i == upper_iters2-1: + result2[:, -latent_lenght:] = samples_i + else: + result2[:, i*latent_lenght:(i+1)*latent_lenght] = samples_i + x_minus1 = samples_i[:, -1:, ...] # Take last slice for next iteration + + result2 = result2.permute(0,2,1,3,4) + x_result2 = torch.zeros((3,slice_num2,512,512)) + dec_unit = 65 + dec_latent_unit=17 + num_dec_iter = slice_num2 // dec_unit + 1 if slice_num2 % dec_unit != 0 else slice_num2 // dec_unit + + for i in range(num_dec_iter): + if i == num_dec_iter - 1: + remaining_slices = slice_num2 % dec_unit if slice_num2 % dec_unit != 0 else dec_unit + x_result2[:, -remaining_slices:] = model.decode_first_stage(result2[:, :, -latent_lenght:])[0][:, -remaining_slices:] + else: + x_result2[:, i*dec_unit:(i+1)*dec_unit] = model.decode_first_stage(result2[:, :, i*latent_lenght:(i+1)*latent_lenght])[0] + + x_result2 = x_result2*2 + x_result2[x_result2>1.0] = 1.0 + x_result2[x_result2<-1.0] = -1.0 + x_result2 = (x_result2+1)/2 + x_result2_ = x_result2.mean(axis=0).detach().cpu().numpy() + x_result2 = x_result2_.transpose(1,2,0) + + # Compute fusion boundaries based on result segments + boundaries = compute_fusion_boundaries( + upper_iters=upper_iters, + latent_length=16, + decode_ratio=4, + blend_width=16 + ) + + # Fuse volumes: start with vol1, replace regions around boundaries with vol2 + print("Fusing volumes: replacing boundary regions with vol2...") + x_result_fused = alternating_volume_fusion_with_boundaries( + x_result, + x_result2, + boundaries, + offset=32, + replace_width=16 + ) + + + + # Load reference and save fused result + ref_root = '/sd/shuhan/CT-RATE/dataset/valid_fixed' + ref_nii = os.path.join(ref_root, name.split('_')[0]+'_'+name.split('_')[1], name.split('_')[0]+'_'+name.split('_')[1]+'_'+name.split('_')[2],name+'.nii.gz') + + if os.path.exists(ref_nii): + pass + else: + ref_root = '/sd/shuhan/CT-RATE/dataset/train_fixed' + ref_nii = os.path.join(ref_root, name.split('_')[0]+'_'+name.split('_')[1], name.split('_')[0]+'_'+name.split('_')[1]+'_'+name.split('_')[2],name+'.nii.gz') + + + affine = nib.load(ref_nii).affine + + x_result_fused = x_result_fused*2000.0 - 1000.0 + # data_path = os.path.join(save_path, str(f'{name}_{disease_label}.nii.gz').replace('/', '_')) + + + data_nii = nib.Nifti1Image(x_result_fused.astype(np.int16), affine) + + nib.save(data_nii, data_path) + print(f"Saved fused volume: {data_path}") + + # disease_root = '/sd/shuhan/CT-RATE/seg_rxg' + # disease_path = os.path.join(disease_root, str(f'{name}.nii.gz')) + # nii_img = nib.load(str(disease_path)) + # disease_data = nii_img.get_fdata()[int(disease_mask_channel[0])] + # data_nii = nib.Nifti1Image(disease_data.astype(np.uint16), affine) + # nib.save(data_nii, os.path.join(save_path, str(f'{name}_disease_{disease_mask_channel[0]}.nii.gz'))) + + # breakpoint() \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/inference_valid.json b/Abnormal-CT-Generation-MultiDisease/inference_valid.json new file mode 100644 index 0000000000000000000000000000000000000000..6a24d44dca377823b240f1ff920d66ef6909d6f1 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/inference_valid.json @@ -0,0 +1,193 @@ +{"volume_path": "dataset/train_fixed/train_1441/train_1441_a/train_1441_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1441_a_2.nii.gz", "disease_mask": "seg_rxg/train_1441_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "4 mm non-contouring opacity with a minor fissure in the anterior segment of the upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1441/train_1441_a/train_1441_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1441_a_2.nii.gz", "disease_mask": "seg_rxg/train_1441_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectasis in the inferior segment of the left lung lingula", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1653/train_1653_a/train_1653_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1653_a_2.nii.gz", "disease_mask": "seg_rxg/train_1653_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral localized ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_2218/train_2218_a/train_2218_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2218_a_2.nii.gz", "disease_mask": "seg_rxg/train_2218_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodule in the superior segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1765/train_1765_a/train_1765_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_1765_a_1.nii.gz", "disease_mask": "seg_rxg/train_1765_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the lower lobe and upper lobe of both lungs and the middle lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1765/train_1765_a/train_1765_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_1765_a_1.nii.gz", "disease_mask": "seg_rxg/train_1765_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidations in the lower lobe and upper lobe of both lungs and the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1765/train_1765_a/train_1765_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_1765_a_1.nii.gz", "disease_mask": "seg_rxg/train_1765_a_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Findings more prominent in the lower lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1765/train_1765_a/train_1765_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_1765_a_1.nii.gz", "disease_mask": "seg_rxg/train_1765_a_1.nii.gz", "disease_mask_channel": "3", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_1474/train_1474_a/train_1474_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1474_a_2.nii.gz", "disease_mask": "seg_rxg/train_1474_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectasis in the central parts of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1474/train_1474_a/train_1474_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1474_a_2.nii.gz", "disease_mask": "seg_rxg/train_1474_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_1474/train_1474_a/train_1474_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1474_a_2.nii.gz", "disease_mask": "seg_rxg/train_1474_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1431/train_1431_a/train_1431_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1431_a_2.nii.gz", "disease_mask": "seg_rxg/train_1431_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripherally located ground glass opacities in the upper and lower lobes of the left lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1431/train_1431_a/train_1431_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1431_a_2.nii.gz", "disease_mask": "seg_rxg/train_1431_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Peripherally located ground glass opacities in the upper and middle lobes of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1431/train_1431_a/train_1431_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1431_a_2.nii.gz", "disease_mask": "seg_rxg/train_1431_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities are more prominent in the left lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_3498/train_3498_a/train_3498_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3498_a_2.nii.gz", "disease_mask": "seg_rxg/train_3498_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Mild emphysematous change, more prominent in the upper lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_2906/train_2906_a/train_2906_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_2906_a_1.nii.gz", "disease_mask": "seg_rxg/train_2906_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Tree-in-bud appearance in the right lung upper lobe anterior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2906/train_2906_a/train_2906_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_2906_a_1.nii.gz", "disease_mask": "seg_rxg/train_2906_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Tree-in-bud appearance in the right lung upper lobe posterior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2906/train_2906_a/train_2906_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_2906_a_1.nii.gz", "disease_mask": "seg_rxg/train_2906_a_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Micronodules", "disease_findings": "Tree-in-bud appearance in the right lung lower lobe superior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2906/train_2906_a/train_2906_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_2906_a_1.nii.gz", "disease_mask": "seg_rxg/train_2906_a_1.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2906/train_2906_a/train_2906_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_2906_a_1.nii.gz", "disease_mask": "seg_rxg/train_2906_a_1.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest nodule measuring 6.5 mm in the lateral segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5730/train_5730_a/train_5730_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_5730_a_1.nii.gz", "disease_mask": "seg_rxg/train_5730_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules in both lungs, the largest measuring approximately 10 mm in diameter in the lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5730/train_5730_a/train_5730_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_5730_a_1.nii.gz", "disease_mask": "seg_rxg/train_5730_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities around some of the nodules", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6800/train_6800_a/train_6800_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_6800_a_1.nii.gz", "disease_mask": "seg_rxg/train_6800_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific nodule measuring 2-3 mm in diameter in the laterobasal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6139/train_6139_a/train_6139_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_6139_a_1.nii.gz", "disease_mask": "seg_rxg/train_6139_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Mosaic attenuation patterns in both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_6139/train_6139_a/train_6139_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_6139_a_1.nii.gz", "disease_mask": "seg_rxg/train_6139_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Thickening of interlobular septa in both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_6139/train_6139_a/train_6139_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_6139_a_1.nii.gz", "disease_mask": "seg_rxg/train_6139_a_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Slight patchy ground-glass density in the superior segment of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5679/train_5679_a/train_5679_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5679_a_2.nii.gz", "disease_mask": "seg_rxg/train_5679_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Patchy, peripherally located ground glass densities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the right upper lobe posterior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the right lower lobe superior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the right middle lobe medial segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the left upper lobe anterior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the left upper lobe lingular segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the left lower lobe superior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the left lower lobe posterobasal segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "7", "disease_label": "Bronchiectasis", "disease_findings": "Cystic bronchiectasis and peribronchial wall thickening in the left lower lobe mediobasal segment", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_6983/train_6983_a/train_6983_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6983_a_2.nii.gz", "disease_mask": "seg_rxg/train_6983_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "7 mm ground-glass opacity with a surrounding halo in the basal segment of the lower lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6983/train_6983_a/train_6983_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6983_a_2.nii.gz", "disease_mask": "seg_rxg/train_6983_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Slight ground-glass opacities at the apicoposterior level of the upper lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6803/train_6803_a/train_6803_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6803_a_2.nii.gz", "disease_mask": "seg_rxg/train_6803_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Three peripheral nodules measuring 10.5 mm in the apicoposterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6803/train_6803_a/train_6803_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6803_a_2.nii.gz", "disease_mask": "seg_rxg/train_6803_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule measuring 11.5 mm in the laterobasal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6803/train_6803_a/train_6803_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6803_a_2.nii.gz", "disease_mask": "seg_rxg/train_6803_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule measuring 5 mm in the laterobasal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6803/train_6803_a/train_6803_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6803_a_2.nii.gz", "disease_mask": "seg_rxg/train_6803_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacity in the left lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6551/train_6551_a/train_6551_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6551_a_2.nii.gz", "disease_mask": "seg_rxg/train_6551_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Paraseptal emphysematous changes in the posterior segments of the superior segments of the lower lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_6551/train_6551_a/train_6551_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6551_a_2.nii.gz", "disease_mask": "seg_rxg/train_6551_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Passive atelectatic changes in the medial segment of the right middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6551/train_6551_a/train_6551_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6551_a_2.nii.gz", "disease_mask": "seg_rxg/train_6551_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Pleuroparenchymal fibroatelectatic changes in the inferior lingular segment of the left upper lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6551/train_6551_a/train_6551_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6551_a_2.nii.gz", "disease_mask": "seg_rxg/train_6551_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Bronchial wall thickening", "disease_findings": "Segmental and subsegmental peribronchial thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodular lesion identified as the primary mass in the peripheral-subpleural area of the medial superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodular lesion measuring approximately 21 mm at its widest point", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear density increases and minimal structural distortion in the vicinity of the described lesion", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodule in the superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Micronodules", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Atelectasis, consolidation", "disease_findings": "Localized atelectasis in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7209/train_7209_a/train_7209_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_7209_a_1.nii.gz", "disease_mask": "seg_rxg/train_7209_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Bilateral pleural effusion measuring 1.5 cm on the right at its deepest point", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_7209/train_7209_a/train_7209_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_7209_a_1.nii.gz", "disease_mask": "seg_rxg/train_7209_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular consolidations in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7209/train_7209_a/train_7209_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_7209_a_1.nii.gz", "disease_mask": "seg_rxg/train_7209_a_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of increased density in air bronchograms", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7331/train_7331_a/train_7331_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7331_a_2.nii.gz", "disease_mask": "seg_rxg/train_7331_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Centriacinar nodular infiltrates in the right lower lobe basal segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7331/train_7331_a/train_7331_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7331_a_2.nii.gz", "disease_mask": "seg_rxg/train_7331_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities surrounding centriacinar nodular infiltrates in the right lower lobe basal segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Occasional linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with air bronchogram in the laterobasal segment of the right lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Surrounding ground glass opacity in the laterobasal segment of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Groundglass opacity", "disease_findings": "Nodular ground glass opacities in the superior segment of both lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Groundglass opacity", "disease_findings": "Nodular ground glass opacities in the posterior segment of the right upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_12714/train_12714_a/train_12714_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12714_a_2.nii.gz", "disease_mask": "seg_rxg/train_12714_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Two subcentimeter nonspecific parenchymal nodules in the superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_12017/train_12017_a/train_12017_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_12017_a_1.nii.gz", "disease_mask": "seg_rxg/train_12017_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Minimal reticulonodular density increases in the apex of both lungs, consistent with scarring", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11150/train_11150_b/train_11150_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11150_b_2.nii.gz", "disease_mask": "seg_rxg/train_11150_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "14x17 mm nodule in the superior segment of the lower lobe of the right lung with peripheral ground glass opacity and spiculated margins", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11150/train_11150_b/train_11150_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11150_b_2.nii.gz", "disease_mask": "seg_rxg/train_11150_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of subsegmental atelectasis in the right lung middle lobe medial segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_11150/train_11150_b/train_11150_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11150_b_2.nii.gz", "disease_mask": "seg_rxg/train_11150_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of subsegmental atelectasis in the left lung upper lobe lingular segment inferior subsegment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_682/train_682_a/train_682_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_682_a_2.nii.gz", "disease_mask": "seg_rxg/train_682_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with air bronchograms in the superior-posterior basal segments of the lower lobes", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_682/train_682_a/train_682_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_682_a_2.nii.gz", "disease_mask": "seg_rxg/train_682_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities with indistinct borders in the lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_595/train_595_a/train_595_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_595_a_1.nii.gz", "disease_mask": "seg_rxg/train_595_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Bilateral basal segments of the lower lobes with patchy ground-glass opacities and interstitial expansion", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_428/train_428_a/train_428_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_428_a_2.nii.gz", "disease_mask": "seg_rxg/train_428_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_428/train_428_a/train_428_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_428_a_2.nii.gz", "disease_mask": "seg_rxg/train_428_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13082/train_13082_a/train_13082_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13082_a_1.nii.gz", "disease_mask": "seg_rxg/train_13082_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Focal consolidative density at the paramediastinal level of the right middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_13082/train_13082_a/train_13082_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13082_a_1.nii.gz", "disease_mask": "seg_rxg/train_13082_a_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Focal scarring changes at the posterobasal level of the left lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13082/train_13082_a/train_13082_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13082_a_1.nii.gz", "disease_mask": "seg_rxg/train_13082_a_1.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Calcified nodule measuring 3 mm at the laterobasal level", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13177/train_13177_a/train_13177_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13177_a_2.nii.gz", "disease_mask": "seg_rxg/train_13177_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectatic changes causing mild structural distortion in the left lung inferior lingular segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_13301/train_13301_a/train_13301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13301_a_2.nii.gz", "disease_mask": "seg_rxg/train_13301_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter calcific nodule in the upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13301/train_13301_a/train_13301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13301_a_2.nii.gz", "disease_mask": "seg_rxg/train_13301_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in the upper lobe of the left lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13301/train_13301_a/train_13301_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13301_a_2.nii.gz", "disease_mask": "seg_rxg/train_13301_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13113/train_13113_a/train_13113_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13113_a_1.nii.gz", "disease_mask": "seg_rxg/train_13113_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "A few subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13119/train_13119_c/train_13119_c_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13119_c_1.nii.gz", "disease_mask": "seg_rxg/train_13119_c_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Interlobar septal thickening in both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_13119/train_13119_c/train_13119_c_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13119_c_1.nii.gz", "disease_mask": "seg_rxg/train_13119_c_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13119/train_13119_c/train_13119_c_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13119_c_1.nii.gz", "disease_mask": "seg_rxg/train_13119_c_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidations in the lower lobe of the left lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_13119/train_13119_c/train_13119_c_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13119_c_1.nii.gz", "disease_mask": "seg_rxg/train_13119_c_1.nii.gz", "disease_mask_channel": "3", "disease_label": "Pleural effusion or thickening", "disease_findings": "Minimal bilateral pleural effusion", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_13417/train_13417_d/train_13417_d_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13417_d_1.nii.gz", "disease_mask": "seg_rxg/train_13417_d_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Stable 3 mm subcentimeter nodule near the subpleural area in the laterobasal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13417/train_13417_d/train_13417_d_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13417_d_1.nii.gz", "disease_mask": "seg_rxg/train_13417_d_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Fibrotic subcentimeter densities in the lower lobe of the right lung", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13076/train_13076_a/train_13076_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13076_a_1.nii.gz", "disease_mask": "seg_rxg/train_13076_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Fibrotic densities in the upper lobes of both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13076/train_13076_a/train_13076_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13076_a_1.nii.gz", "disease_mask": "seg_rxg/train_13076_a_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13111/train_13111_a/train_13111_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13111_a_2.nii.gz", "disease_mask": "seg_rxg/train_13111_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Smooth interlobular septal thickening in the peripheral areas, more prominent in the lower lobes of both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_13075/train_13075_a/train_13075_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13075_a_2.nii.gz", "disease_mask": "seg_rxg/train_13075_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Atelectatic changes at the basal levels of both lower lobes", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_13440/train_13440_a/train_13440_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13440_a_1.nii.gz", "disease_mask": "seg_rxg/train_13440_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13479/train_13479_j/train_13479_j_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13479_j_1.nii.gz", "disease_mask": "seg_rxg/train_13479_j_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Parenchymal scarring in the right lower lobe posterobasal segment", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13479/train_13479_j/train_13479_j_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13479_j_1.nii.gz", "disease_mask": "seg_rxg/train_13479_j_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Minimal ground-glass opacities in the right lower lobe posterobasal segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_13447/train_13447_a/train_13447_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13447_a_1.nii.gz", "disease_mask": "seg_rxg/train_13447_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter non-specific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13577/train_13577_a/train_13577_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13577_a_2.nii.gz", "disease_mask": "seg_rxg/train_13577_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Focal ground glass opacity in the laterobasal segment of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_13430/train_13430_a/train_13430_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13430_a_2.nii.gz", "disease_mask": "seg_rxg/train_13430_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Pleural-based nodule measuring 5x3 mm in the superior segment of the lower lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13497/train_13497_a/train_13497_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13497_a_1.nii.gz", "disease_mask": "seg_rxg/train_13497_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Linear atelectasis in the medial segment of the right middle lobe", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13431/train_13431_a/train_13431_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13431_a_1.nii.gz", "disease_mask": "seg_rxg/train_13431_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Patchy ground-glass opacity in the apicoposterior segment of the left upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_13020/train_13020_a/train_13020_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13020_a_2.nii.gz", "disease_mask": "seg_rxg/train_13020_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "6x5 mm semisolid nodule in the posterobasal segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13020/train_13020_a/train_13020_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13020_a_2.nii.gz", "disease_mask": "seg_rxg/train_13020_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Subcentimeter pure calcific nonspecific nodules in both lung parenchyma", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_13278/train_13278_b/train_13278_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13278_b_2.nii.gz", "disease_mask": "seg_rxg/train_13278_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "2-3 nodules in the right lung, largest measuring approximately 4 mm in diameter", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13278/train_13278_b/train_13278_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13278_b_2.nii.gz", "disease_mask": "seg_rxg/train_13278_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the laterobasal segment of the right middle and lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_13148/train_13148_a/train_13148_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13148_a_2.nii.gz", "disease_mask": "seg_rxg/train_13148_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Two subpleural subcentimeter nodules measuring 4 mm at the basal level in the superior right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13148/train_13148_a/train_13148_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13148_a_2.nii.gz", "disease_mask": "seg_rxg/train_13148_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Two subpleural subcentimeter nodules measuring 4 mm in the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13249/train_13249_b/train_13249_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13249_b_2.nii.gz", "disease_mask": "seg_rxg/train_13249_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular infiltration in the peripheral subpleural areas", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_13249/train_13249_b/train_13249_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_13249_b_2.nii.gz", "disease_mask": "seg_rxg/train_13249_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Areas of increased density consistent with linear atelectasis in the basal segments of the lower lobes", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_13339/train_13339_a/train_13339_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13339_a_1.nii.gz", "disease_mask": "seg_rxg/train_13339_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific pulmonary nodule, 3.9 mm, in the anterobasal segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_13339/train_13339_a/train_13339_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_13339_a_1.nii.gz", "disease_mask": "seg_rxg/train_13339_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific pulmonary nodule, 4.3 mm, in the laterobasal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7331/train_7331_a/train_7331_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7331_a_2.nii.gz", "disease_mask": "seg_rxg/train_7331_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Centriacinar nodular infiltrates in the right lower lobe basal segment", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7331/train_7331_a/train_7331_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7331_a_2.nii.gz", "disease_mask": "seg_rxg/train_7331_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities surrounding centriacinar nodular infiltrates in the right lower lobe basal segment", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_12714/train_12714_a/train_12714_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_12714_a_2.nii.gz", "disease_mask": "seg_rxg/train_12714_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Two subcentimeter nonspecific parenchymal nodules in the superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_682/train_682_a/train_682_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_682_a_2.nii.gz", "disease_mask": "seg_rxg/train_682_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with air bronchograms in the superior-posterior basal segments of the lower lobes", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_682/train_682_a/train_682_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_682_a_2.nii.gz", "disease_mask": "seg_rxg/train_682_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities with indistinct borders in the lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1474/train_1474_a/train_1474_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1474_a_2.nii.gz", "disease_mask": "seg_rxg/train_1474_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Bronchiectasis", "disease_findings": "Minimal bronchiectasis in the central parts of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_1474/train_1474_a/train_1474_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1474_a_2.nii.gz", "disease_mask": "seg_rxg/train_1474_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_1474/train_1474_a/train_1474_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1474_a_2.nii.gz", "disease_mask": "seg_rxg/train_1474_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6803/train_6803_a/train_6803_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6803_a_2.nii.gz", "disease_mask": "seg_rxg/train_6803_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Three peripheral nodules measuring 10.5 mm in the apicoposterior segment of the left upper lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6803/train_6803_a/train_6803_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6803_a_2.nii.gz", "disease_mask": "seg_rxg/train_6803_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule measuring 11.5 mm in the laterobasal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6803/train_6803_a/train_6803_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6803_a_2.nii.gz", "disease_mask": "seg_rxg/train_6803_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodule measuring 5 mm in the laterobasal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6803/train_6803_a/train_6803_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6803_a_2.nii.gz", "disease_mask": "seg_rxg/train_6803_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacity in the left lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6139/train_6139_a/train_6139_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_6139_a_1.nii.gz", "disease_mask": "seg_rxg/train_6139_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Septal thickening", "disease_findings": "Mosaic attenuation patterns in both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_6139/train_6139_a/train_6139_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_6139_a_1.nii.gz", "disease_mask": "seg_rxg/train_6139_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Septal thickening", "disease_findings": "Thickening of interlobular septa in both lungs", "disease_class": 162, "disease_label_text": "Septal thickening"} +{"volume_path": "dataset/train_fixed/train_6139/train_6139_a/train_6139_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_6139_a_1.nii.gz", "disease_mask": "seg_rxg/train_6139_a_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Slight patchy ground-glass density in the superior segment of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_3498/train_3498_a/train_3498_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_3498_a_2.nii.gz", "disease_mask": "seg_rxg/train_3498_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Mild emphysematous change, more prominent in the upper lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodular lesion identified as the primary mass in the peripheral-subpleural area of the medial superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodular lesion measuring approximately 21 mm at its widest point", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Linear", "disease_findings": "Linear density increases and minimal structural distortion in the vicinity of the described lesion", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nodule in the superior segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Micronodules", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_7250/train_7250_i/train_7250_i_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_7250_i_2.nii.gz", "disease_mask": "seg_rxg/train_7250_i_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Atelectasis, consolidation", "disease_findings": "Localized atelectasis in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1653/train_1653_a/train_1653_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1653_a_2.nii.gz", "disease_mask": "seg_rxg/train_1653_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripheral localized ground glass opacities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/valid_fixed/valid_1170/valid_1170_c/valid_1170_c_1.nii.gz", "organ_mask": "organ_mask_whole/valid/valid_1170_c_1.nii.gz", "disease_mask": "seg_rxg/valid_1170_c_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Diffuse emphysematous changes throughout both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/valid_fixed/valid_1170/valid_1170_c/valid_1170_c_1.nii.gz", "organ_mask": "organ_mask_whole/valid/valid_1170_c_1.nii.gz", "disease_mask": "seg_rxg/valid_1170_c_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Bronchiectasis", "disease_findings": "Cylindrical bronchiectasis with peribronchial thickening in the lower lobes of both lungs", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/valid_fixed/valid_1170/valid_1170_c/valid_1170_c_1.nii.gz", "organ_mask": "organ_mask_whole/valid/valid_1170_c_1.nii.gz", "disease_mask": "seg_rxg/valid_1170_c_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Bronchiectasis", "disease_findings": "Cylindrical bronchiectasis with peribronchial thickening in the middle lobe of the right lung", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/valid_fixed/valid_1170/valid_1170_c/valid_1170_c_1.nii.gz", "organ_mask": "organ_mask_whole/valid/valid_1170_c_1.nii.gz", "disease_mask": "seg_rxg/valid_1170_c_1.nii.gz", "disease_mask_channel": "3", "disease_label": "Bronchiectasis", "disease_findings": "Cystic bronchiectasis in the lower lobes", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/valid_fixed/valid_1170/valid_1170_c/valid_1170_c_1.nii.gz", "organ_mask": "organ_mask_whole/valid/valid_1170_c_1.nii.gz", "disease_mask": "seg_rxg/valid_1170_c_1.nii.gz", "disease_mask_channel": "5", "disease_label": "Bronchiectasis", "disease_findings": "Bronchiectasis associated with soft tissue density changes in the upper lobe of the right lung", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/valid_fixed/valid_1170/valid_1170_c/valid_1170_c_1.nii.gz", "organ_mask": "organ_mask_whole/valid/valid_1170_c_1.nii.gz", "disease_mask": "seg_rxg/valid_1170_c_1.nii.gz", "disease_mask_channel": "6", "disease_label": "Bronchiectasis", "disease_findings": "Bronchiectasis associated with structural distortion in the upper lobe of the right lung", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/valid_fixed/valid_1170/valid_1170_c/valid_1170_c_1.nii.gz", "organ_mask": "organ_mask_whole/valid/valid_1170_c_1.nii.gz", "disease_mask": "seg_rxg/valid_1170_c_1.nii.gz", "disease_mask_channel": "7", "disease_label": "Micronodules", "disease_findings": "Widespread tree-in-bud opacities in both lungs", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_12017/train_12017_a/train_12017_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_12017_a_1.nii.gz", "disease_mask": "seg_rxg/train_12017_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Linear", "disease_findings": "Minimal reticulonodular density increases in the apex of both lungs, consistent with scarring", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_6800/train_6800_a/train_6800_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_6800_a_1.nii.gz", "disease_mask": "seg_rxg/train_6800_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nonspecific nodule measuring 2-3 mm in diameter in the laterobasal segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2906/train_2906_a/train_2906_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_2906_a_1.nii.gz", "disease_mask": "seg_rxg/train_2906_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Tree-in-bud appearance in the right lung upper lobe anterior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2906/train_2906_a/train_2906_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_2906_a_1.nii.gz", "disease_mask": "seg_rxg/train_2906_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Tree-in-bud appearance in the right lung upper lobe posterior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2906/train_2906_a/train_2906_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_2906_a_1.nii.gz", "disease_mask": "seg_rxg/train_2906_a_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Micronodules", "disease_findings": "Tree-in-bud appearance in the right lung lower lobe superior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_2906/train_2906_a/train_2906_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_2906_a_1.nii.gz", "disease_mask": "seg_rxg/train_2906_a_1.nii.gz", "disease_mask_channel": "3", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_2906/train_2906_a/train_2906_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_2906_a_1.nii.gz", "disease_mask": "seg_rxg/train_2906_a_1.nii.gz", "disease_mask_channel": "4", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Largest nodule measuring 6.5 mm in the lateral segment of the right lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1431/train_1431_a/train_1431_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1431_a_2.nii.gz", "disease_mask": "seg_rxg/train_1431_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Peripherally located ground glass opacities in the upper and lower lobes of the left lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1431/train_1431_a/train_1431_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1431_a_2.nii.gz", "disease_mask": "seg_rxg/train_1431_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Peripherally located ground glass opacities in the upper and middle lobes of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1431/train_1431_a/train_1431_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1431_a_2.nii.gz", "disease_mask": "seg_rxg/train_1431_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Groundglass opacity", "disease_findings": "Ground glass opacities are more prominent in the left lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_7209/train_7209_a/train_7209_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_7209_a_1.nii.gz", "disease_mask": "seg_rxg/train_7209_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Pleural effusion or thickening", "disease_findings": "Bilateral pleural effusion measuring 1.5 cm on the right at its deepest point", "disease_class": 159, "disease_label_text": "Pleural effusion or thickening"} +{"volume_path": "dataset/train_fixed/train_7209/train_7209_a/train_7209_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_7209_a_1.nii.gz", "disease_mask": "seg_rxg/train_7209_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Nodular consolidations in both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_7209/train_7209_a/train_7209_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_7209_a_1.nii.gz", "disease_mask": "seg_rxg/train_7209_a_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of increased density in air bronchograms", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_595/train_595_a/train_595_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_595_a_1.nii.gz", "disease_mask": "seg_rxg/train_595_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Bilateral basal segments of the lower lobes with patchy ground-glass opacities and interstitial expansion", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_428/train_428_a/train_428_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_428_a_2.nii.gz", "disease_mask": "seg_rxg/train_428_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_428/train_428_a/train_428_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_428_a_2.nii.gz", "disease_mask": "seg_rxg/train_428_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodules in both lungs", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11150/train_11150_b/train_11150_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11150_b_2.nii.gz", "disease_mask": "seg_rxg/train_11150_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "14x17 mm nodule in the superior segment of the lower lobe of the right lung with peripheral ground glass opacity and spiculated margins", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_11150/train_11150_b/train_11150_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11150_b_2.nii.gz", "disease_mask": "seg_rxg/train_11150_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of subsegmental atelectasis in the right lung middle lobe medial segment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_11150/train_11150_b/train_11150_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11150_b_2.nii.gz", "disease_mask": "seg_rxg/train_11150_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Areas of subsegmental atelectasis in the left lung upper lobe lingular segment inferior subsegment", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_5679/train_5679_a/train_5679_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_5679_a_2.nii.gz", "disease_mask": "seg_rxg/train_5679_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Patchy, peripherally located ground glass densities in both lungs", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6983/train_6983_a/train_6983_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6983_a_2.nii.gz", "disease_mask": "seg_rxg/train_6983_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "7 mm ground-glass opacity with a surrounding halo in the basal segment of the lower lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_6983/train_6983_a/train_6983_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6983_a_2.nii.gz", "disease_mask": "seg_rxg/train_6983_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Slight ground-glass opacities at the apicoposterior level of the upper lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1441/train_1441_a/train_1441_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1441_a_2.nii.gz", "disease_mask": "seg_rxg/train_1441_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "4 mm non-contouring opacity with a minor fissure in the anterior segment of the upper lobe of the right lung", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_1441/train_1441_a/train_1441_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_1441_a_2.nii.gz", "disease_mask": "seg_rxg/train_1441_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Subsegmental atelectasis in the inferior segment of the left lung lingula", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6551/train_6551_a/train_6551_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6551_a_2.nii.gz", "disease_mask": "seg_rxg/train_6551_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Paraseptal emphysematous changes in the posterior segments of the superior segments of the lower lobes of both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_6551/train_6551_a/train_6551_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6551_a_2.nii.gz", "disease_mask": "seg_rxg/train_6551_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Passive atelectatic changes in the medial segment of the right middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6551/train_6551_a/train_6551_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6551_a_2.nii.gz", "disease_mask": "seg_rxg/train_6551_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Pleuroparenchymal fibroatelectatic changes in the inferior lingular segment of the left upper lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_6551/train_6551_a/train_6551_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6551_a_2.nii.gz", "disease_mask": "seg_rxg/train_6551_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Bronchial wall thickening", "disease_findings": "Segmental and subsegmental peribronchial thickening in both lungs", "disease_class": 151, "disease_label_text": "Bronchial wall thickening"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Linear", "disease_findings": "Occasional linear atelectasis in both lungs", "disease_class": 156, "disease_label_text": "Linear"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidation with air bronchogram in the laterobasal segment of the right lower lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Groundglass opacity", "disease_findings": "Surrounding ground glass opacity in the laterobasal segment of the right lower lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Groundglass opacity", "disease_findings": "Nodular ground glass opacities in the superior segment of both lower lobes", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_11195/train_11195_a/train_11195_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_11195_a_2.nii.gz", "disease_mask": "seg_rxg/train_11195_a_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Groundglass opacity", "disease_findings": "Nodular ground glass opacities in the posterior segment of the right upper lobe", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_5730/train_5730_a/train_5730_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_5730_a_1.nii.gz", "disease_mask": "seg_rxg/train_5730_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Nodules in both lungs, the largest measuring approximately 10 mm in diameter in the lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_5730/train_5730_a/train_5730_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_5730_a_1.nii.gz", "disease_mask": "seg_rxg/train_5730_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities around some of the nodules", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1765/train_1765_a/train_1765_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_1765_a_1.nii.gz", "disease_mask": "seg_rxg/train_1765_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Groundglass opacity", "disease_findings": "Ground-glass opacities in the lower lobe and upper lobe of both lungs and the middle lobe of the right lung", "disease_class": 154, "disease_label_text": "Groundglass opacity"} +{"volume_path": "dataset/train_fixed/train_1765/train_1765_a/train_1765_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_1765_a_1.nii.gz", "disease_mask": "seg_rxg/train_1765_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Consolidations in the lower lobe and upper lobe of both lungs and the middle lobe of the right lung", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1765/train_1765_a/train_1765_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_1765_a_1.nii.gz", "disease_mask": "seg_rxg/train_1765_a_1.nii.gz", "disease_mask_channel": "2", "disease_label": "Atelectasis, consolidation", "disease_findings": "Findings more prominent in the lower lobes of both lungs", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/train_fixed/train_1765/train_1765_a/train_1765_a_1.nii.gz", "organ_mask": "organ_mask_whole/train/train_1765_a_1.nii.gz", "disease_mask": "seg_rxg/train_1765_a_1.nii.gz", "disease_mask_channel": "3", "disease_label": "Emphysema", "disease_findings": "Minimal emphysematous changes in both lungs", "disease_class": 153, "disease_label_text": "Emphysema"} +{"volume_path": "dataset/valid_fixed/valid_915/valid_915_b/valid_915_b_2.nii.gz", "organ_mask": "organ_mask_whole/valid/valid_915_b_2.nii.gz", "disease_mask": "seg_rxg/valid_915_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter subpleural nodule at the junction of the anteromedial and lateral segments of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/valid_fixed/valid_915/valid_915_b/valid_915_b_2.nii.gz", "organ_mask": "organ_mask_whole/valid/valid_915_b_2.nii.gz", "disease_mask": "seg_rxg/valid_915_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter subpleural irregularities at the right apical level", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the right upper lobe posterior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "1", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the right lower lobe superior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "2", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the right middle lobe medial segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "3", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the left upper lobe anterior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "4", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the left upper lobe lingular segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "5", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the left lower lobe superior segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "6", "disease_label": "Micronodules", "disease_findings": "Centriacinar nodular densities with a tree-in-bud appearance in the left lower lobe posterobasal segment", "disease_class": 157, "disease_label_text": "Micronodules"} +{"volume_path": "dataset/train_fixed/train_6520/train_6520_b/train_6520_b_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_6520_b_2.nii.gz", "disease_mask": "seg_rxg/train_6520_b_2.nii.gz", "disease_mask_channel": "7", "disease_label": "Bronchiectasis", "disease_findings": "Cystic bronchiectasis and peribronchial wall thickening in the left lower lobe mediobasal segment", "disease_class": 152, "disease_label_text": "Bronchiectasis"} +{"volume_path": "dataset/train_fixed/train_2218/train_2218_a/train_2218_a_2.nii.gz", "organ_mask": "organ_mask_whole/train/train_2218_a_2.nii.gz", "disease_mask": "seg_rxg/train_2218_a_2.nii.gz", "disease_mask_channel": "0", "disease_label": "Pulmonary nodules/masses", "disease_findings": "Subcentimeter nonspecific nodule in the superior segment of the left lower lobe", "disease_class": 161, "disease_label_text": "Pulmonary nodules/masses"} +{"volume_path": "dataset/valid_fixed/valid_991/valid_991_a/valid_991_a_1.nii.gz", "organ_mask": "organ_mask_whole/valid/valid_991_a_1.nii.gz", "disease_mask": "seg_rxg/valid_991_a_1.nii.gz", "disease_mask_channel": "0", "disease_label": "Atelectasis, consolidation", "disease_findings": "Mild atelectatic changes in the right middle lobe", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} +{"volume_path": "dataset/valid_fixed/valid_991/valid_991_a/valid_991_a_1.nii.gz", "organ_mask": "organ_mask_whole/valid/valid_991_a_1.nii.gz", "disease_mask": "seg_rxg/valid_991_a_1.nii.gz", "disease_mask_channel": "1", "disease_label": "Atelectasis, consolidation", "disease_findings": "Mild atelectatic changes in the left upper lobe's inferior lingula", "disease_class": 150, "disease_label_text": "Atelectasis, consolidation"} diff --git a/Abnormal-CT-Generation-MultiDisease/inference_valid.py b/Abnormal-CT-Generation-MultiDisease/inference_valid.py new file mode 100644 index 0000000000000000000000000000000000000000..610af6e9e346a375c46b93542a03eb6b2eaf201d --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/inference_valid.py @@ -0,0 +1,385 @@ +import glob +import nibabel as nib +import numpy as np +import os +import torch + +from einops import rearrange +from omegaconf import OmegaConf +from torch.utils.data import DataLoader +from tqdm import tqdm + +from ldm.util import instantiate_from_config +import argparse + + +import os +import re + +def fix_filename(filepath): + """ + 修复包含特殊字符的文件路径,将斜杠等特殊字符替换为下划线 + + Args: + filepath: 原始文件路径 + + Returns: + 修复后的文件路径 + """ + # 分离目录和文件名 + directory = os.path.dirname(filepath) + filename = os.path.basename(filepath) + + # 替换文件名中的特殊字符为下划线 + # 包括:斜杠、反斜杠、冒号、星号、问号、引号、尖括号、竖线 + clean_filename = re.sub(r'[/\\:*?"<>|]', '_', filename) + + # 处理连续的下划线,替换为单个下划线 + clean_filename = re.sub(r'_+', '_', clean_filename) + + # 去除首尾的下划线 + clean_filename = clean_filename.strip('_') + + # 组合新路径 + if directory: + return os.path.join(directory, clean_filename) + else: + return clean_filename + + + +def compute_orientation(init_axcodes, final_axcodes): + """ + A thin wrapper around ``nib.orientations.ornt_transform`` + + :param init_axcodes: Initial orientation codes + :param final_axcodes: Target orientation codes + :return: orientations array, start_ornt, end_ornt + """ + ornt_init = nib.orientations.axcodes2ornt(init_axcodes) + ornt_fin = nib.orientations.axcodes2ornt(final_axcodes) + + ornt_transf = nib.orientations.ornt_transform(ornt_init, ornt_fin) + + return ornt_transf, ornt_init, ornt_fin + +def do_reorientation(data_array, init_axcodes, final_axcodes): + """ + source: https://niftynet.readthedocs.io/en/dev/_modules/niftynet/io/misc_io.html#do_reorientation + Performs the reorientation (changing order of axes) + + :param data_array: 3D Array to reorient + :param init_axcodes: Initial orientation + :param final_axcodes: Target orientation + :return data_reoriented: New data array in its reoriented form + """ + ornt_transf, ornt_init, ornt_fin = compute_orientation(init_axcodes, final_axcodes) + if np.array_equal(ornt_init, ornt_fin): + return data_array + + return nib.orientations.apply_orientation(data_array, ornt_transf) + +def compute_fusion_boundaries(upper_iters, latent_length, decode_ratio=4, blend_width=16): + """ + Compute fusion boundaries based on result segments + + :param upper_iters: Number of iterations in the generation loop + :param latent_length: Length of latent dimension (17) + :param decode_ratio: Ratio between decoded and latent dimensions (4) + :param blend_width: Width of blending region on each side (16 slices) + :return: List of boundary dictionaries for fusion + """ + boundaries = [] + + for i in range(1, upper_iters): + # Calculate boundary position in latent space + latent_boundary = i * latent_length + + # Convert to decoded space (multiply by decode_ratio) + decoded_boundary = latent_boundary * decode_ratio + 1 + + # Define blending region: boundary ± blend_width + blend_start = max(0, decoded_boundary - blend_width) + blend_end = decoded_boundary + blend_width + + boundaries.append({ + 'boundary': decoded_boundary, + 'blend_start': blend_start, + 'blend_end': blend_end + }) + + return boundaries + + +def alternating_volume_fusion_with_boundaries(vol1, vol2, boundaries, offset=32, replace_width=16): + """ + Fuse two volumes by replacing regions around boundaries + + :param vol1: First volume with shape (x, y, z) - used as base + :param vol2: Second volume with shape (x, y, z-offset), offset by 32 slices + :param boundaries: List of boundary dictionaries from compute_fusion_boundaries + :param offset: Offset between vol1 and vol2 (default 32) + :param replace_width: Width on each side of boundary to replace (default 16) + :return: Fused volume with vol2 replacing vol1 at boundary regions + """ + x, y, total_slices = vol1.shape + fused = vol1.copy() # Start with vol1 as base + + # Sort boundaries by position + sorted_boundaries = sorted(boundaries, key=lambda b: b['boundary']) + + print(f"Fusion starting with vol1 as base") + + for i, boundary_info in enumerate(sorted_boundaries): + boundary = boundary_info['boundary'] + + # Define replacement region: boundary ± replace_width + replace_start = max(0, boundary - replace_width) + replace_end = min(boundary + replace_width, total_slices) + + # Map to vol2 coordinates + vol2_start = replace_start - offset + vol2_end = replace_end - offset + + # Replace this region with vol2 if valid + if vol2_start >= 0 and vol2_end <= vol2.shape[2]: + fused[:, :, replace_start:replace_end] = vol2[:, :, vol2_start:vol2_end] + print(f" Boundary {i+1}: z={boundary}, replaced z[{replace_start}:{replace_end}] with vol2[{vol2_start}:{vol2_end}]") + else: + print(f" Boundary {i+1}: z={boundary}, skipped (vol2 out of range)") + + print(f"Fusion complete with {len(sorted_boundaries)} boundaries") + + return fused + +parser = argparse.ArgumentParser() +parser.add_argument( + "-t", + "--time_steps", + type=int, + default=20, +) +args = parser.parse_args() +ddim_steps=args.time_steps + +logdir = 'logs/full_ct_3d_with_body_mask2' +ckpt = os.path.join(logdir, "checkpoints", "epoch=000061.ckpt") + +configs_file = "configs/latent-diffusion/infer_valid.yaml" +configs = OmegaConf.load(configs_file) +model = instantiate_from_config(configs.model) +model.init_from_ckpt(ckpt) +model.eval() + +device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") +print("Using device", device) +model = model.to(device) + +config = OmegaConf.load('./configs/latent-diffusion/infer_valid.yaml') +data = instantiate_from_config(config.data) +data.prepare_data() +data.setup() + +save_path = f'inference_valid' + +if not os.path.exists(save_path): + os.makedirs(save_path) + +val_dataset = data.datasets['validation'] +batch_size = 1 +valloader = DataLoader(val_dataset, batch_size=batch_size, shuffle=False, num_workers=2, pin_memory=True) +val_num = len(val_dataset) +save_gt = True + +for idx, data in tqdm(enumerate(valloader)): + + if idx >= val_num: + break + + name=data['name'][0].split('.')[0] + volume_data = data['volume_data'] + volume_seg = data['volume_seg'] + disease_mask_channel=data['disease_mask_channel'] + disease_label=data['disease_label'][0] + # breakpoint() + + data_path = os.path.join(save_path, str(f'{name}_disease_{disease_mask_channel[0]}_{disease_label}.nii.gz').replace('/', '_')) + if os.path.exists(data_path): + print(f"File already exists, skipping: {data_path}") + continue + + print(volume_data.shape) + window_length = 65 + latent_lenght=17 + slice_num = volume_data.shape[2] + + upper_iters = slice_num // window_length + 1 if slice_num % window_length != 0 else slice_num // window_length + result = torch.zeros((batch_size, upper_iters*latent_lenght, 16, 64, 64)).cuda() + print('upper_iters', upper_iters) + + for i in range(upper_iters): + print('i', i) + input_data={} + if i == upper_iters-1: + input_data['name'] = data['name'] + input_data['volume_data'] = data['volume_data'][:, :, -window_length:].to(device) + input_data['volume_seg'] = data['volume_seg'][:, :, -window_length:].to(device) + input_data['input_text'] = data['input_text'] + else: + input_data['volume_data'] = data['volume_data'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['volume_seg'] = data['volume_seg'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['input_text'] = data['input_text'] + + with torch.no_grad(): + _, c = model.get_input(input_data, model.first_stage_key) + + if i == 0: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps) + else: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps, previous=x_minus1) + + samples_i = rearrange(samples_i, '(b z) c h w -> b z c h w', z=latent_lenght) + + if i == upper_iters-1: + result[:, -latent_lenght:] = samples_i + else: + result[:, i*latent_lenght:(i+1)*latent_lenght] = samples_i + x_minus1 = samples_i[:, -1:, ...] # Take last slice for next iteration + + result = result.permute(0,2,1,3,4) + x_result = torch.zeros((3,slice_num,512,512)) + dec_unit = 65 + dec_latent_unit=17 + num_dec_iter = slice_num // dec_unit + 1 if slice_num % dec_unit != 0 else slice_num // dec_unit + + for i in range(num_dec_iter): + if i == num_dec_iter - 1: + remaining_slices = slice_num % dec_unit if slice_num % dec_unit != 0 else dec_unit + x_result[:, -remaining_slices:] = model.decode_first_stage(result[:, :, -latent_lenght:])[0][:, -remaining_slices:] + else: + x_result[:, i*dec_unit:(i+1)*dec_unit] = model.decode_first_stage(result[:, :, i*latent_lenght:(i+1)*latent_lenght])[0] + + x_result = x_result*2 + x_result[x_result>1.0] = 1.0 + x_result[x_result<-1.0] = -1.0 + x_result = (x_result+1)/2 + x_result_ = x_result.mean(axis=0).detach().cpu().numpy() + x_result = x_result_.transpose(1,2,0) + + # Process second volume with offset + data2={} + data2['volume_data'] = data['volume_data'][:, :, 32:] + data2['volume_seg'] = data['volume_seg'][:, :, 32:] + data2['name'] = data['name'] + data2['input_text'] = data['input_text'] + + volume_data = data2['volume_data'] + volume_seg = data2['volume_seg'] + + window_length = 65 + latent_lenght=17 + slice_num2 = volume_data.shape[2] + + upper_iters2 = slice_num2 // window_length + 1 if slice_num2 % window_length != 0 else slice_num2 // window_length + result2 = torch.zeros((batch_size, upper_iters2*latent_lenght, 16, 64, 64)).cuda() + print('upper_iters2', upper_iters2) + + for i in range(upper_iters2): + print('i', i) + input_data={} + if i == upper_iters2-1: + input_data['name'] = data2['name'] + input_data['volume_data'] = data2['volume_data'][:, :, -window_length:].to(device) + input_data['volume_seg'] = data2['volume_seg'][:, :, -window_length:].to(device) + input_data['input_text'] = data2['input_text'] + else: + input_data['volume_data'] = data2['volume_data'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['volume_seg'] = data2['volume_seg'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['input_text'] = data2['input_text'] + + with torch.no_grad(): + _, c = model.get_input(input_data, model.first_stage_key) + + if i == 0: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps) + else: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps, previous=x_minus1) + + samples_i = rearrange(samples_i, '(b z) c h w -> b z c h w', z=latent_lenght) + + if i == upper_iters2-1: + result2[:, -latent_lenght:] = samples_i + else: + result2[:, i*latent_lenght:(i+1)*latent_lenght] = samples_i + x_minus1 = samples_i[:, -1:, ...] # Take last slice for next iteration + + result2 = result2.permute(0,2,1,3,4) + x_result2 = torch.zeros((3,slice_num2,512,512)) + dec_unit = 65 + dec_latent_unit=17 + num_dec_iter = slice_num2 // dec_unit + 1 if slice_num2 % dec_unit != 0 else slice_num2 // dec_unit + + for i in range(num_dec_iter): + if i == num_dec_iter - 1: + remaining_slices = slice_num2 % dec_unit if slice_num2 % dec_unit != 0 else dec_unit + x_result2[:, -remaining_slices:] = model.decode_first_stage(result2[:, :, -latent_lenght:])[0][:, -remaining_slices:] + else: + x_result2[:, i*dec_unit:(i+1)*dec_unit] = model.decode_first_stage(result2[:, :, i*latent_lenght:(i+1)*latent_lenght])[0] + + x_result2 = x_result2*2 + x_result2[x_result2>1.0] = 1.0 + x_result2[x_result2<-1.0] = -1.0 + x_result2 = (x_result2+1)/2 + x_result2_ = x_result2.mean(axis=0).detach().cpu().numpy() + x_result2 = x_result2_.transpose(1,2,0) + + # Compute fusion boundaries based on result segments + boundaries = compute_fusion_boundaries( + upper_iters=upper_iters, + latent_length=16, + decode_ratio=4, + blend_width=16 + ) + + # Fuse volumes: start with vol1, replace regions around boundaries with vol2 + print("Fusing volumes: replacing boundary regions with vol2...") + x_result_fused = alternating_volume_fusion_with_boundaries( + x_result, + x_result2, + boundaries, + offset=32, + replace_width=16 + ) + + + + # Load reference and save fused result + ref_root = '/sd/shuhan/CT-RATE/dataset/valid_fixed' + ref_nii = os.path.join(ref_root, name.split('_')[0]+'_'+name.split('_')[1], name.split('_')[0]+'_'+name.split('_')[1]+'_'+name.split('_')[2],name+'.nii.gz') + + if os.path.exists(ref_nii): + pass + else: + ref_root = '/sd/shuhan/CT-RATE/dataset/train_fixed' + ref_nii = os.path.join(ref_root, name.split('_')[0]+'_'+name.split('_')[1], name.split('_')[0]+'_'+name.split('_')[1]+'_'+name.split('_')[2],name+'.nii.gz') + + + affine = nib.load(ref_nii).affine + + x_result_fused = x_result_fused*2000.0 - 1000.0 + # data_path = os.path.join(save_path, str(f'{name}_{disease_label}.nii.gz').replace('/', '_')) + + + data_nii = nib.Nifti1Image(x_result_fused.astype(np.int16), affine) + + nib.save(data_nii, data_path) + print(f"Saved fused volume: {data_path}") + + # disease_root = '/sd/shuhan/CT-RATE/seg_rxg' + # disease_path = os.path.join(disease_root, str(f'{name}.nii.gz')) + # nii_img = nib.load(str(disease_path)) + # disease_data = nii_img.get_fdata()[int(disease_mask_channel[0])] + # data_nii = nib.Nifti1Image(disease_data.astype(np.uint16), affine) + # nib.save(data_nii, os.path.join(save_path, str(f'{name}_disease_{disease_mask_channel[0]}.nii.gz'))) + + # breakpoint() \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/inference_valid_multi.py b/Abnormal-CT-Generation-MultiDisease/inference_valid_multi.py new file mode 100644 index 0000000000000000000000000000000000000000..01d3c7cdd8c0ff29b1b8a2f8cbdc96bf1a827b3b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/inference_valid_multi.py @@ -0,0 +1,385 @@ +import glob +import nibabel as nib +import numpy as np +import os +import torch + +from einops import rearrange +from omegaconf import OmegaConf +from torch.utils.data import DataLoader +from tqdm import tqdm + +from ldm.util import instantiate_from_config +import argparse + + +import os +import re + +def fix_filename(filepath): + """ + 修复包含特殊字符的文件路径,将斜杠等特殊字符替换为下划线 + + Args: + filepath: 原始文件路径 + + Returns: + 修复后的文件路径 + """ + # 分离目录和文件名 + directory = os.path.dirname(filepath) + filename = os.path.basename(filepath) + + # 替换文件名中的特殊字符为下划线 + # 包括:斜杠、反斜杠、冒号、星号、问号、引号、尖括号、竖线 + clean_filename = re.sub(r'[/\\:*?"<>|]', '_', filename) + + # 处理连续的下划线,替换为单个下划线 + clean_filename = re.sub(r'_+', '_', clean_filename) + + # 去除首尾的下划线 + clean_filename = clean_filename.strip('_') + + # 组合新路径 + if directory: + return os.path.join(directory, clean_filename) + else: + return clean_filename + + + +def compute_orientation(init_axcodes, final_axcodes): + """ + A thin wrapper around ``nib.orientations.ornt_transform`` + + :param init_axcodes: Initial orientation codes + :param final_axcodes: Target orientation codes + :return: orientations array, start_ornt, end_ornt + """ + ornt_init = nib.orientations.axcodes2ornt(init_axcodes) + ornt_fin = nib.orientations.axcodes2ornt(final_axcodes) + + ornt_transf = nib.orientations.ornt_transform(ornt_init, ornt_fin) + + return ornt_transf, ornt_init, ornt_fin + +def do_reorientation(data_array, init_axcodes, final_axcodes): + """ + source: https://niftynet.readthedocs.io/en/dev/_modules/niftynet/io/misc_io.html#do_reorientation + Performs the reorientation (changing order of axes) + + :param data_array: 3D Array to reorient + :param init_axcodes: Initial orientation + :param final_axcodes: Target orientation + :return data_reoriented: New data array in its reoriented form + """ + ornt_transf, ornt_init, ornt_fin = compute_orientation(init_axcodes, final_axcodes) + if np.array_equal(ornt_init, ornt_fin): + return data_array + + return nib.orientations.apply_orientation(data_array, ornt_transf) + +def compute_fusion_boundaries(upper_iters, latent_length, decode_ratio=4, blend_width=16): + """ + Compute fusion boundaries based on result segments + + :param upper_iters: Number of iterations in the generation loop + :param latent_length: Length of latent dimension (17) + :param decode_ratio: Ratio between decoded and latent dimensions (4) + :param blend_width: Width of blending region on each side (16 slices) + :return: List of boundary dictionaries for fusion + """ + boundaries = [] + + for i in range(1, upper_iters): + # Calculate boundary position in latent space + latent_boundary = i * latent_length + + # Convert to decoded space (multiply by decode_ratio) + decoded_boundary = latent_boundary * decode_ratio + 1 + + # Define blending region: boundary ± blend_width + blend_start = max(0, decoded_boundary - blend_width) + blend_end = decoded_boundary + blend_width + + boundaries.append({ + 'boundary': decoded_boundary, + 'blend_start': blend_start, + 'blend_end': blend_end + }) + + return boundaries + + +def alternating_volume_fusion_with_boundaries(vol1, vol2, boundaries, offset=32, replace_width=16): + """ + Fuse two volumes by replacing regions around boundaries + + :param vol1: First volume with shape (x, y, z) - used as base + :param vol2: Second volume with shape (x, y, z-offset), offset by 32 slices + :param boundaries: List of boundary dictionaries from compute_fusion_boundaries + :param offset: Offset between vol1 and vol2 (default 32) + :param replace_width: Width on each side of boundary to replace (default 16) + :return: Fused volume with vol2 replacing vol1 at boundary regions + """ + x, y, total_slices = vol1.shape + fused = vol1.copy() # Start with vol1 as base + + # Sort boundaries by position + sorted_boundaries = sorted(boundaries, key=lambda b: b['boundary']) + + print(f"Fusion starting with vol1 as base") + + for i, boundary_info in enumerate(sorted_boundaries): + boundary = boundary_info['boundary'] + + # Define replacement region: boundary ± replace_width + replace_start = max(0, boundary - replace_width) + replace_end = min(boundary + replace_width, total_slices) + + # Map to vol2 coordinates + vol2_start = replace_start - offset + vol2_end = replace_end - offset + + # Replace this region with vol2 if valid + if vol2_start >= 0 and vol2_end <= vol2.shape[2]: + fused[:, :, replace_start:replace_end] = vol2[:, :, vol2_start:vol2_end] + print(f" Boundary {i+1}: z={boundary}, replaced z[{replace_start}:{replace_end}] with vol2[{vol2_start}:{vol2_end}]") + else: + print(f" Boundary {i+1}: z={boundary}, skipped (vol2 out of range)") + + print(f"Fusion complete with {len(sorted_boundaries)} boundaries") + + return fused + +parser = argparse.ArgumentParser() +parser.add_argument( + "-t", + "--time_steps", + type=int, + default=20, +) +args = parser.parse_args() +ddim_steps=args.time_steps + +logdir = 'logs/full_ct_3d_with_body_mask2' +ckpt = os.path.join(logdir, "checkpoints", "epoch=000095.ckpt") + +configs_file = "configs/latent-diffusion/infer_valid_multi.yaml" +configs = OmegaConf.load(configs_file) +model = instantiate_from_config(configs.model) +model.init_from_ckpt(ckpt) +model.eval() + +device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") +print("Using device", device) +model = model.to(device) + +config = OmegaConf.load('./configs/latent-diffusion/infer_valid_multi.yaml') +data = instantiate_from_config(config.data) +data.prepare_data() +data.setup() + +save_path = f'inference_valid' + +if not os.path.exists(save_path): + os.makedirs(save_path) + +val_dataset = data.datasets['validation'] +batch_size = 1 +valloader = DataLoader(val_dataset, batch_size=batch_size, shuffle=False, num_workers=2, pin_memory=True) +val_num = len(val_dataset) +save_gt = True + +for idx, data in tqdm(enumerate(valloader)): + + if idx >= val_num: + break + + name=data['name'][0].split('.')[0] + volume_data = data['volume_data'] + volume_seg = data['volume_seg'] + # disease_mask_channel=data['disease_mask_channel'] + # disease_label=data['disease_label'][0] + # breakpoint() + + data_path = os.path.join(save_path, str(f'{name}.nii.gz').replace('/', '_')) + if os.path.exists(data_path): + print(f"File already exists, skipping: {data_path}") + continue + + print(volume_data.shape) + window_length = 65 + latent_lenght=17 + slice_num = volume_data.shape[2] + + upper_iters = slice_num // window_length + 1 if slice_num % window_length != 0 else slice_num // window_length + result = torch.zeros((batch_size, upper_iters*latent_lenght, 16, 64, 64)).cuda() + print('upper_iters', upper_iters) + + for i in range(upper_iters): + print('i', i) + input_data={} + if i == upper_iters-1: + input_data['name'] = data['name'] + input_data['volume_data'] = data['volume_data'][:, :, -window_length:].to(device) + input_data['volume_seg'] = data['volume_seg'][:, :, -window_length:].to(device) + input_data['input_text'] = data['input_text'] + else: + input_data['volume_data'] = data['volume_data'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['volume_seg'] = data['volume_seg'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['input_text'] = data['input_text'] + + with torch.no_grad(): + _, c = model.get_input(input_data, model.first_stage_key) + + if i == 0: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps) + else: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps, previous=x_minus1) + + samples_i = rearrange(samples_i, '(b z) c h w -> b z c h w', z=latent_lenght) + + if i == upper_iters-1: + result[:, -latent_lenght:] = samples_i + else: + result[:, i*latent_lenght:(i+1)*latent_lenght] = samples_i + x_minus1 = samples_i[:, -1:, ...] # Take last slice for next iteration + + result = result.permute(0,2,1,3,4) + x_result = torch.zeros((3,slice_num,512,512)) + dec_unit = 65 + dec_latent_unit=17 + num_dec_iter = slice_num // dec_unit + 1 if slice_num % dec_unit != 0 else slice_num // dec_unit + + for i in range(num_dec_iter): + if i == num_dec_iter - 1: + remaining_slices = slice_num % dec_unit if slice_num % dec_unit != 0 else dec_unit + x_result[:, -remaining_slices:] = model.decode_first_stage(result[:, :, -latent_lenght:])[0][:, -remaining_slices:] + else: + x_result[:, i*dec_unit:(i+1)*dec_unit] = model.decode_first_stage(result[:, :, i*latent_lenght:(i+1)*latent_lenght])[0] + + x_result = x_result*2 + x_result[x_result>1.0] = 1.0 + x_result[x_result<-1.0] = -1.0 + x_result = (x_result+1)/2 + x_result_ = x_result.mean(axis=0).detach().cpu().numpy() + x_result = x_result_.transpose(1,2,0) + + # Process second volume with offset + data2={} + data2['volume_data'] = data['volume_data'][:, :, 32:] + data2['volume_seg'] = data['volume_seg'][:, :, 32:] + data2['name'] = data['name'] + data2['input_text'] = data['input_text'] + + volume_data = data2['volume_data'] + volume_seg = data2['volume_seg'] + + window_length = 65 + latent_lenght=17 + slice_num2 = volume_data.shape[2] + + upper_iters2 = slice_num2 // window_length + 1 if slice_num2 % window_length != 0 else slice_num2 // window_length + result2 = torch.zeros((batch_size, upper_iters2*latent_lenght, 16, 64, 64)).cuda() + print('upper_iters2', upper_iters2) + + for i in range(upper_iters2): + print('i', i) + input_data={} + if i == upper_iters2-1: + input_data['name'] = data2['name'] + input_data['volume_data'] = data2['volume_data'][:, :, -window_length:].to(device) + input_data['volume_seg'] = data2['volume_seg'][:, :, -window_length:].to(device) + input_data['input_text'] = data2['input_text'] + else: + input_data['volume_data'] = data2['volume_data'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['volume_seg'] = data2['volume_seg'][:, :, i*window_length:(i+1)*window_length].to(device) + input_data['input_text'] = data2['input_text'] + + with torch.no_grad(): + _, c = model.get_input(input_data, model.first_stage_key) + + if i == 0: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps) + else: + samples_i, _ = model.sample_log(cond=c, batch_size=latent_lenght, ddim=True, eta=1., ddim_steps=ddim_steps, previous=x_minus1) + + samples_i = rearrange(samples_i, '(b z) c h w -> b z c h w', z=latent_lenght) + + if i == upper_iters2-1: + result2[:, -latent_lenght:] = samples_i + else: + result2[:, i*latent_lenght:(i+1)*latent_lenght] = samples_i + x_minus1 = samples_i[:, -1:, ...] # Take last slice for next iteration + + result2 = result2.permute(0,2,1,3,4) + x_result2 = torch.zeros((3,slice_num2,512,512)) + dec_unit = 65 + dec_latent_unit=17 + num_dec_iter = slice_num2 // dec_unit + 1 if slice_num2 % dec_unit != 0 else slice_num2 // dec_unit + + for i in range(num_dec_iter): + if i == num_dec_iter - 1: + remaining_slices = slice_num2 % dec_unit if slice_num2 % dec_unit != 0 else dec_unit + x_result2[:, -remaining_slices:] = model.decode_first_stage(result2[:, :, -latent_lenght:])[0][:, -remaining_slices:] + else: + x_result2[:, i*dec_unit:(i+1)*dec_unit] = model.decode_first_stage(result2[:, :, i*latent_lenght:(i+1)*latent_lenght])[0] + + x_result2 = x_result2*2 + x_result2[x_result2>1.0] = 1.0 + x_result2[x_result2<-1.0] = -1.0 + x_result2 = (x_result2+1)/2 + x_result2_ = x_result2.mean(axis=0).detach().cpu().numpy() + x_result2 = x_result2_.transpose(1,2,0) + + # Compute fusion boundaries based on result segments + boundaries = compute_fusion_boundaries( + upper_iters=upper_iters, + latent_length=16, + decode_ratio=4, + blend_width=16 + ) + + # Fuse volumes: start with vol1, replace regions around boundaries with vol2 + print("Fusing volumes: replacing boundary regions with vol2...") + x_result_fused = alternating_volume_fusion_with_boundaries( + x_result, + x_result2, + boundaries, + offset=32, + replace_width=16 + ) + + + + # Load reference and save fused result + ref_root = '/sd/shuhan/CT-RATE/dataset/valid_fixed' + ref_nii = os.path.join(ref_root, name.split('_')[0]+'_'+name.split('_')[1], name.split('_')[0]+'_'+name.split('_')[1]+'_'+name.split('_')[2],name+'.nii.gz') + + if os.path.exists(ref_nii): + pass + else: + ref_root = '/sd/shuhan/CT-RATE/dataset/train_fixed' + ref_nii = os.path.join(ref_root, name.split('_')[0]+'_'+name.split('_')[1], name.split('_')[0]+'_'+name.split('_')[1]+'_'+name.split('_')[2],name+'.nii.gz') + + + affine = nib.load(ref_nii).affine + + x_result_fused = x_result_fused*2000.0 - 1000.0 + # data_path = os.path.join(save_path, str(f'{name}_{disease_label}.nii.gz').replace('/', '_')) + + + data_nii = nib.Nifti1Image(x_result_fused.astype(np.int16), affine) + + nib.save(data_nii, data_path) + print(f"Saved fused volume: {data_path}") + + # disease_root = '/sd/shuhan/CT-RATE/seg_rxg' + # disease_path = os.path.join(disease_root, str(f'{name}.nii.gz')) + # nii_img = nib.load(str(disease_path)) + # disease_data = nii_img.get_fdata()[int(disease_mask_channel[0])] + # data_nii = nib.Nifti1Image(disease_data.astype(np.uint16), affine) + # nib.save(data_nii, os.path.join(save_path, str(f'{name}_disease_{disease_mask_channel[0]}.nii.gz'))) + + # breakpoint() \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..4fceca14f5fb1df9ea3c574aa34171b7535f9c1e Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..c1415ef9f94356ad2dedc5f744ce736ce872517f Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..96e0ac2a8e7def359ff3af8e7f65b9312577fe5e Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..b4a82be4c1a0ade3d7793db554770edc0bfdd68d Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/__pycache__/util.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__init__.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..4d585c5fed0f81b0c81a0df0b32934f29043c3bf Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..dcae1cc2a155b4669450636f0c6431b409262a43 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..8c6854e4985da7553d59e54ee49a86df6de25deb Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..962eb525db1281df73d4944b31648debc4c2f8b9 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/__init__.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..f62127f43bf9a884b478a30ee940de0cafba5416 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..62f10fa49484060f6c2529fdd078b4f8f5350222 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..8613aefbbece2be667c6928efc9bbc079a7d6f6f Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..d5f8d946c7cc582924c401460a0bd1d53ba17b3b Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/base.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..7ce55f78c5702287b305c30ee676246ea5f77919 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..05ca7e02c6a2bcdb566a2640e3b28233a74ad376 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..8fdaf9935528fcce888d5bd9baf37a0024ecf5da Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..92e69215c79e5bfd074960cf1723d7de24ec575e Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference_eval.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference_eval.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..c455cd4514bb4109ec132bdab4827234006c53e2 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference_eval.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference_eval.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference_eval.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..20a6f679d384e7531d18a1d4188caf492fa16326 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_inference_eval.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_train.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_train.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..30c76c3ad1a43775c97094c9e946092ff6354dd2 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_train.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_train.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_train.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..4175e0c7b83627944ec6a35e7315a779af327477 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_train.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_train.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_train.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..829556961ef058fbf22a38bb9345be8b7b34ed55 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/ct_clip_data_train.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..ed383f6fd9c95c3c7469958670bfc7354ed9843d Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_28class.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_28class.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..1907a2c3f7e622bfbd30ed4d1695ae84d6a05fe2 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_28class.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_28class_randomclass.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_28class_randomclass.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..8540131846ede266575be8b28cf01613ea708a73 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_28class_randomclass.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_28class_randomclass_kl4.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_28class_randomclass_kl4.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..e9882abf3a382bac957126a313357c8e71fbe076 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_28class_randomclass_kl4.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_resize.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_resize.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..f7d0fba4c518af0b3301a66c4927dd453a8731be Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_abdomenatlas_resize.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_infer.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_infer.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..4f1a0012cb67644b03ecf1e84a4bb91e38220d60 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_infer.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_infer_28class_randomclass.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_infer_28class_randomclass.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..713c768d44b86eb6daace82ae15860befec32ff2 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_infer_28class_randomclass.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_resize.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_resize.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..8fd615e3cb140d3415099665880401c4c71c594b Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/slice_dataset_resize.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..b7a0f67b0431fdf0c298f0c0ec005907d613caf7 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_13class_randomclass.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_13class_randomclass.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..caeaee882bb9e97f249cb917220d7777d049b3e0 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_13class_randomclass.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_28class_randomclass.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_28class_randomclass.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..d13b4f6de861140cbccb08d6d90e0a0afd93a459 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_28class_randomclass.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_infer.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_infer.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..896c3e3c012ba432feb5514623797bc129705c77 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_infer.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_infer.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_infer.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..7bb6423a19965a0d23fbb13b7c6f8391860e9f89 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/data/__pycache__/volume_dataset_infer.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/base.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/base.py new file mode 100644 index 0000000000000000000000000000000000000000..8d0b0a1a55a9b58d716ce7caeaa041070a77546d --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/base.py @@ -0,0 +1,23 @@ +from abc import abstractmethod +from torch.utils.data import Dataset, ConcatDataset, ChainDataset, IterableDataset + + +class Txt2ImgIterableBaseDataset(IterableDataset): + ''' + Define an interface to make the IterableDatasets for text2img data chainable + ''' + def __init__(self, num_records=0, valid_ids=None, size=256): + super().__init__() + self.num_records = num_records + self.valid_ids = valid_ids + self.sample_ids = valid_ids + self.size = size + + print(f'{self.__class__.__name__} dataset contains {self.__len__()} examples.') + + def __len__(self): + return self.num_records + + @abstractmethod + def __iter__(self): + pass diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/ct_clip_data_inference.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/ct_clip_data_inference.py new file mode 100644 index 0000000000000000000000000000000000000000..62c9de40b7ccc3a2fb96ea8bc535b4cebb2279f4 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/ct_clip_data_inference.py @@ -0,0 +1,428 @@ +import os +import glob +import json +import torch +import pandas as pd +import numpy as np +from PIL import Image +from torch.utils.data import Dataset +import torchvision.transforms as transforms +from functools import partial +import torch.nn.functional as F +import tqdm +import nibabel as nib +import copy +from collections import defaultdict + +def merge_list_elements(data): + """ + data: list of lists,每个元素长度为7 + 返回: 合并后的list,仍然是7列 + """ + merged = defaultdict(lambda: [[] for _ in range(5)]) + + for item in data: + key = (item[0], item[1]) # 前两个元素作为key + for i in range(5): # 后面五列分别合并 + merged[key][i].append(item[2+i]) + + # 重新组装结果 + result = [] + for (a, b), rest in merged.items(): + result.append([a, b] + rest) + return result + + +def build_multiclass_mask( + disease_files, + disease_mask_channels, + disease_class_ids, + *, + channel_axis=0, + threshold=0.5, + out_dtype=np.float32, +): + """ + 将 (file, channel, class_id) 成对列表合并为单通道多类别mask(无重叠假设)。 + 背景=0,其它为对应 class_id。 + + 参数 + ---- + disease_files : List[str] + NIfTI 文件路径列表 + disease_mask_channels : List[int] + 通道索引列表(与 disease_files 一一对应) + disease_class_ids : List[int] + 每个 (file, channel) 对应的最终类别 id + channel_axis : int + 通道所在维度(默认 0) + threshold : float + 二值化阈值;mask > threshold 视为前景 + out_dtype : numpy dtype + 输出标签图的数据类型,默认 uint16 + """ + + if not (len(disease_files) == len(disease_mask_channels) == len(disease_class_ids)): + raise ValueError("disease_files、disease_mask_channels、disease_class_ids 长度必须一致") + + # 用第一份确定空间形状 + first_img = nib.load(str(disease_files[0])) + first_data = first_img.get_fdata(dtype=np.float32) + sample_mask = np.take(first_data, int(disease_mask_channels[0]), axis=channel_axis) + spatial_shape = sample_mask.shape + + label_map = np.zeros(spatial_shape, dtype=out_dtype) + + for fpath, ch, cid in zip(disease_files, disease_mask_channels, disease_class_ids): + img = nib.load(str(fpath)) + data = img.get_fdata(dtype=np.float32) + # breakpoint() + mask = np.take(data, int(ch), axis=channel_axis) + + if mask.shape != spatial_shape: + raise ValueError( + f"{fpath} 的 mask 形状 {mask.shape} 与期望 {spatial_shape} 不一致,请先重采样" + ) + + # 二值化并赋类标 + label_map[mask > threshold] = cid + + return label_map + +def resize_array(array, current_spacing): + """ + Resize the array to match the target spacing. + + Args: + array (torch.Tensor): Input array to be resized. + current_spacing (tuple): Current voxel spacing (z_spacing, xy_spacing, xy_spacing). + target_spacing (tuple): Target voxel spacing (target_z_spacing, target_x_spacing, target_y_spacing). + + Returns: + np.ndarray: Resized array. + """ + # Calculate new dimensions + original_shape = array.shape[2:] + + + new_shape = [original_shape[0], 512, 512] + scaling_factors = [new_shape[i] / original_shape[i] for i in range(len(original_shape))] + resized_spacing = [current_spacing[i] / scaling_factors[i] for i in range(len(original_shape))] + # Resize the array + resized_array = F.interpolate(array, size=new_shape, mode='trilinear', align_corners=False).cpu().numpy() + # breakpoint() + return resized_array, resized_spacing + +def resize_mask(array, current_spacing): + """ + Resize the array to match the target spacing. + + Args: + array (torch.Tensor): Input array to be resized. + current_spacing (tuple): Current voxel spacing (z_spacing, xy_spacing, xy_spacing). + target_spacing (tuple): Target voxel spacing (target_z_spacing, target_x_spacing, target_y_spacing). + + Returns: + np.ndarray: Resized array. + """ + # Calculate new dimensions + original_shape = array.shape[2:] + + new_shape = [original_shape[0], 512, 512] + + resized_array = F.interpolate(array, size=new_shape, mode='nearest').cpu().numpy() + # breakpoint() + return resized_array + +class CTReportDatasetinfer(Dataset): + def __init__(self, data_folder, csv_file, json_file, min_slices=20, resize_dim=500, force_num_frames=True): + self.data_folder = data_folder + self.min_slices = min_slices + self.json_file=json_file + self.accession_to_text = self.load_accession_text(csv_file) + self.paths=[] + self.samples = self.prepare_samples() + self.transform = transforms.Compose([ + transforms.Resize((resize_dim,resize_dim)), + transforms.ToTensor() + ]) + self.nii_to_tensor = partial(self.nii_img_to_tensor, transform = self.transform) + self.sample_length=65 + + def load_accession_text(self, csv_file): + df = pd.read_csv(csv_file) + accession_to_text = {} + for index, row in df.iterrows(): + accession_to_text[row['VolumeName']] = row["Findings_EN"],row['Impressions_EN'] + return accession_to_text + + + def prepare_samples(self): + samples = [] + # patient_folders = glob.glob(os.path.join(self.data_folder, '*')) + + # Read labels once outside the loop + import json + + with open('/sd/qichen/full_ct_gen/CT-RATE/disease_mask_json/disease_valid_single_prompt_checked_label.json', 'r') as f: + # with open(self.json_file, 'r') as f: + items = [json.loads(line) for line in f] + # breakpoint() + # 2. 提取所有 volume_path + volume_paths = [item['volume_path'] + for item in items + if 'volume_path' in item] + + disease_mask_paths = [item['disease_mask'] + for item in items + if 'disease_mask' in item] + + organ_mask_paths = [item['organ_mask'] + for item in items + if 'organ_mask' in item] + + disease_findings_list = [item['disease_findings'] + for item in items + if 'disease_findings' in item] + disease_mask_channels = [item['disease_mask_channel'] + for item in items + if 'disease_mask_channel' in item] + disease_labels = [item['disease_label'] + for item in items + if 'disease_label' in item] + disease_classes = [item['disease_class'] + for item in items + if 'disease_class' in item] + # breakpoint() + for (ct_file, organ_mask_file, disease_mask_file, disease_findings, disease_mask_channel, disease_label, disease_class) in tqdm.tqdm(zip(volume_paths, organ_mask_paths, disease_mask_paths, disease_findings_list, disease_mask_channels, disease_labels, disease_classes)): + + accession_number = organ_mask_file.split("/")[-1] + + ct_file = '/sd/shuhan/CT-RATE/'+ct_file + organ_file = '/sd/shuhan/CT-RATE/'+organ_mask_file + disease_file = '/sd/shuhan/CT-RATE/'+disease_mask_file + + text_final = disease_findings + + + + samples.append((ct_file, organ_file, disease_file, text_final, disease_mask_channel, disease_label, disease_class)) + + samples = merge_list_elements(samples) + + + with open('/sd/qichen/full_ct_gen/CT-RATE/valid_no_disease_2_checked.json', 'r') as f: + items = [json.loads(line) for line in f] + + # 2. 提取所有 volume_path + volume_paths = [item['volume_path'] + for item in items + if 'volume_path' in item] + organ_masks = [item['organ_mask'] + for item in items + if 'organ_mask' in item] + + for nii_file, organ_mask in tqdm.tqdm(zip(volume_paths, organ_masks)): + + accession_number = nii_file.split("/")[-1] + + seg_file = '/sd/shuhan/CT-RATE/'+organ_mask + nii_file = '/sd/shuhan/CT-RATE/'+nii_file + # accession_number = accession_number.replace(".npz", ".nii.gz") + if accession_number not in self.accession_to_text: + continue + + impression_text = self.accession_to_text[accession_number] + text_final = "" + for text in list(impression_text): + text = str(text) + if text == "Not given.": + text = "" + + text_final = text_final + text + + samples.append((nii_file, seg_file, None, text_final, None, None, None)) + + return samples + + def __len__(self): + return len(self.samples) + + def nii_img_to_tensor(self, ct_file, organ_file, disease_file, disease_mask_channel, disease_label, disease_class, transform): + if disease_file == None: + + nii_img = nib.load(str(ct_file)) + img_data = nii_img.get_fdata() + + df = pd.read_csv("/sd/shuhan/CT-RATE/metadata/all_metadata.csv") #select the metadata + file_name = ct_file.split("/")[-1] + row = df[df['VolumeName'] == file_name] + slope = float(row["RescaleSlope"].iloc[0]) + intercept = float(row["RescaleIntercept"].iloc[0]) + xy_spacing = float(row["XYSpacing"].iloc[0][1:][:-2].split(",")[0]) + z_spacing = float(row["ZSpacing"].iloc[0]) + + nii_seg = nib.load(str(organ_file)) + mask_data = nii_seg.get_fdata() + + current = (z_spacing, xy_spacing, xy_spacing) + + img_data = img_data.transpose(2, 0, 1) + tensor = torch.tensor(img_data) + tensor = tensor.unsqueeze(0).unsqueeze(0) + img_data, target_spacing = resize_array(tensor, current) + img_data = img_data[0][0] + + mask_data = mask_data.transpose(2, 0, 1) + tensor = torch.tensor(mask_data) + tensor = tensor.unsqueeze(0).unsqueeze(0) + mask_data = resize_mask(tensor, current) + mask_data = mask_data[0][0] + # breakpoint() + assert mask_data.shape == img_data.shape + fg_mask = (mask_data>0).astype(np.uint8) + mask_data = (((mask_data ) / 300)).astype(np.float32) * 2 -1 + + hu_min, hu_max = -1000, 1000 + img_data = np.clip(img_data, hu_min, hu_max) + + bg_np = np.ones_like(img_data) * -1000 + + img_data = img_data*fg_mask + bg_np*(1-fg_mask) + + + img_data = (((img_data ) / 1000)).astype(np.float32) + start_id = np.random.randint(0, img_data.shape[0]-self.sample_length) + # img_data = img_data[start_id:start_id+self.sample_length] + # mask_data = mask_data[start_id:start_id+self.sample_length] + + # slices=[] + img_data = img_data/2.0 + mask_data = mask_data/2.0 + + img_data = torch.tensor(img_data) + mask_data = torch.tensor(mask_data) + + + img_data = img_data.unsqueeze(0) + mask_data = mask_data.unsqueeze(0) + + return img_data, mask_data, target_spacing, file_name + else: + ct_nii = nib.load(str(ct_file)) + img_data = ct_nii.get_fdata() + + df = pd.read_csv("/sd/shuhan/CT-RATE/metadata/all_metadata.csv") #select the metadata + file_name = ct_file.split("/")[-1] + row = df[df['VolumeName'] == file_name] + slope = float(row["RescaleSlope"].iloc[0]) + intercept = float(row["RescaleIntercept"].iloc[0]) + xy_spacing = float(row["XYSpacing"].iloc[0][1:][:-2].split(",")[0]) + z_spacing = float(row["ZSpacing"].iloc[0]) + + current = (z_spacing, xy_spacing, xy_spacing) + img_data = img_data.transpose(2, 0, 1) + tensor = torch.tensor(img_data) + tensor = tensor.unsqueeze(0).unsqueeze(0) + img_data, target_spacing = resize_array(tensor, current) + img_data = img_data[0][0] + + organ_nii = nib.load(str(organ_file)) + organ_mask = organ_nii.get_fdata() + organ_mask = organ_mask.transpose(2, 0, 1) + tensor = torch.tensor(organ_mask) + tensor = tensor.unsqueeze(0).unsqueeze(0) + organ_mask = resize_mask(tensor, current) + organ_mask = organ_mask[0][0] + + # disease_nii = nib.load(str(disease_file)) + # disease_mask = disease_nii.get_fdata()[int(disease_mask_channel)] + disease_mask = build_multiclass_mask(disease_file,disease_mask_channel,disease_class) + disease_mask = disease_mask.transpose(2, 0, 1) + tensor = torch.tensor(disease_mask) + tensor = tensor.unsqueeze(0).unsqueeze(0) + disease_mask = resize_mask(tensor, current) + disease_mask = disease_mask[0][0] + + assert organ_mask.shape == img_data.shape + + num_slices = img_data.shape[0] + + # 找到 mask>0 的切片索引 + valid_indices = np.where(disease_mask.sum(axis=(1, 2)) > 0)[0] + + if len(valid_indices) > 0: + # 只保留能完整取 sample_length 的起点 + valid_start_ids = valid_indices[valid_indices <= num_slices - self.sample_length] + + if len(valid_start_ids) > 0: + start_id = np.random.choice(valid_start_ids) + else: + # 如果没有合法起点就退化到随机,但保证能取 sample_length + start_id = np.random.randint(0, num_slices - self.sample_length + 1) + else: + # 如果 mask 全部为0,则完全随机 + start_id = np.random.randint(0, num_slices - self.sample_length + 1) + + end_id = start_id + self.sample_length # 保证长度严格等于 sample_length + # img_data = img_data[start_id:end_id] + # organ_mask = organ_mask[start_id:end_id] + # disease_mask = disease_mask[start_id:end_id] + # img_data = img_data[start_id] + # organ_mask = organ_mask[start_id] + # disease_mask = disease_mask[start_id] + + organ_disease_mask = copy.deepcopy(organ_mask) + # organ_disease_mask[disease_mask>0] = disease_class+100 + organ_disease_mask[disease_mask!=0] = disease_mask[disease_mask != 0]+100 + + organ_mask = (((organ_mask ) / 300)).astype(np.float32) * 2 -1 + organ_disease_mask = (((organ_disease_mask ) / 300)).astype(np.float32) * 2 -1 + + hu_min, hu_max = -1000, 1000 + img_data = np.clip(img_data, hu_min, hu_max) + img_data = (((img_data ) / 1000)).astype(np.float32) + + img_data = torch.tensor(img_data) + organ_mask = torch.tensor(organ_mask) + organ_disease_mask = torch.tensor(organ_disease_mask) + + img_data = img_data/2.0 + organ_mask = organ_mask/2.0 + organ_disease_mask = organ_disease_mask/2.0 + + img_data = img_data.unsqueeze(0) + organ_mask = organ_mask.unsqueeze(0) + organ_disease_mask = organ_disease_mask.unsqueeze(0) + + + # img_data=img_data.repeat(3,1,1,1) + # organ_mask=organ_mask.repeat(3,1,1,1) + # organ_disease_mask=organ_disease_mask.repeat(3,1,1,1) + + + # return img_data, organ_mask, organ_disease_mask, target_spacing, file_name + # return img_data, organ_mask, target_spacing, file_name + return img_data, organ_disease_mask, target_spacing, file_name + + + + def __getitem__(self, index): + ct_file, organ_file, disease_file, input_text, disease_mask_channel, disease_label, disease_class = self.samples[index] + video_tensor, volume_seg, spacing, file_name = self.nii_to_tensor(ct_file, organ_file, disease_file, disease_mask_channel, disease_label, disease_class) + input_text = str(input_text) + input_text = input_text.replace('"', '') + input_text = input_text.replace('\'', '') + input_text = input_text.replace('(', '') + input_text = input_text.replace(')', '') + + # return video_tensor, input_text + example = {} + example['name'] = file_name + example['volume_data'] = video_tensor.float() + example['volume_seg'] = volume_seg.float() + example['spacing'] = spacing + example['input_text'] = input_text + example['disease_mask_channel'] = disease_mask_channel + example['disease_label'] = disease_label + return example \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/ct_clip_data_inference_eval.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/ct_clip_data_inference_eval.py new file mode 100644 index 0000000000000000000000000000000000000000..32e858d9271845190081373c2c8a00f8ced6d04e --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/ct_clip_data_inference_eval.py @@ -0,0 +1,394 @@ +import os +import glob +import json +import torch +import pandas as pd +import numpy as np +from PIL import Image +from torch.utils.data import Dataset +import torchvision.transforms as transforms +from functools import partial +import torch.nn.functional as F +import tqdm +import nibabel as nib +import copy +from collections import defaultdict + +def merge_list_elements(data): + """ + data: list of lists,每个元素长度为7 + 返回: 合并后的list,仍然是7列 + """ + merged = defaultdict(lambda: [[] for _ in range(5)]) + + for item in data: + key = (item[0], item[1]) # 前两个元素作为key + for i in range(5): # 后面五列分别合并 + merged[key][i].append(item[2+i]) + + # 重新组装结果 + result = [] + for (a, b), rest in merged.items(): + result.append([a, b] + rest) + return result + + +def build_multiclass_mask( + disease_files, + disease_mask_channels, + disease_class_ids, + *, + channel_axis=0, + threshold=0.5, + out_dtype=np.float32, +): + """ + 将 (file, channel, class_id) 成对列表合并为单通道多类别mask(无重叠假设)。 + 背景=0,其它为对应 class_id。 + + 参数 + ---- + disease_files : List[str] + NIfTI 文件路径列表 + disease_mask_channels : List[int] + 通道索引列表(与 disease_files 一一对应) + disease_class_ids : List[int] + 每个 (file, channel) 对应的最终类别 id + channel_axis : int + 通道所在维度(默认 0) + threshold : float + 二值化阈值;mask > threshold 视为前景 + out_dtype : numpy dtype + 输出标签图的数据类型,默认 uint16 + """ + + if not (len(disease_files) == len(disease_mask_channels) == len(disease_class_ids)): + raise ValueError("disease_files、disease_mask_channels、disease_class_ids 长度必须一致") + + # 用第一份确定空间形状 + first_img = nib.load(str(disease_files[0])) + first_data = first_img.get_fdata(dtype=np.float32) + sample_mask = np.take(first_data, int(disease_mask_channels[0]), axis=channel_axis) + spatial_shape = sample_mask.shape + + label_map = np.zeros(spatial_shape, dtype=out_dtype) + + for fpath, ch, cid in zip(disease_files, disease_mask_channels, disease_class_ids): + img = nib.load(str(fpath)) + data = img.get_fdata(dtype=np.float32) + # breakpoint() + mask = np.take(data, int(ch), axis=channel_axis) + + if mask.shape != spatial_shape: + raise ValueError( + f"{fpath} 的 mask 形状 {mask.shape} 与期望 {spatial_shape} 不一致,请先重采样" + ) + + # 二值化并赋类标 + label_map[mask > threshold] = cid + + return label_map + +def resize_array(array, current_spacing): + """ + Resize the array to match the target spacing. + + Args: + array (torch.Tensor): Input array to be resized. + current_spacing (tuple): Current voxel spacing (z_spacing, xy_spacing, xy_spacing). + target_spacing (tuple): Target voxel spacing (target_z_spacing, target_x_spacing, target_y_spacing). + + Returns: + np.ndarray: Resized array. + """ + # Calculate new dimensions + original_shape = array.shape[2:] + + + new_shape = [original_shape[0], 512, 512] + scaling_factors = [new_shape[i] / original_shape[i] for i in range(len(original_shape))] + resized_spacing = [current_spacing[i] / scaling_factors[i] for i in range(len(original_shape))] + # Resize the array + resized_array = F.interpolate(array, size=new_shape, mode='trilinear', align_corners=False).cpu().numpy() + # breakpoint() + return resized_array, resized_spacing + +def resize_mask(array, current_spacing): + """ + Resize the array to match the target spacing. + + Args: + array (torch.Tensor): Input array to be resized. + current_spacing (tuple): Current voxel spacing (z_spacing, xy_spacing, xy_spacing). + target_spacing (tuple): Target voxel spacing (target_z_spacing, target_x_spacing, target_y_spacing). + + Returns: + np.ndarray: Resized array. + """ + # Calculate new dimensions + original_shape = array.shape[2:] + + new_shape = [original_shape[0], 512, 512] + + resized_array = F.interpolate(array, size=new_shape, mode='nearest').cpu().numpy() + # breakpoint() + return resized_array + +class CTReportDatasetinfer(Dataset): + def __init__(self, data_folder, csv_file, json_file, min_slices=20, resize_dim=500, force_num_frames=True): + self.data_folder = data_folder + self.min_slices = min_slices + self.json_file=json_file + self.accession_to_text = self.load_accession_text(csv_file) + self.paths=[] + self.samples = self.prepare_samples() + self.transform = transforms.Compose([ + transforms.Resize((resize_dim,resize_dim)), + transforms.ToTensor() + ]) + self.nii_to_tensor = partial(self.nii_img_to_tensor, transform = self.transform) + self.sample_length=65 + + def load_accession_text(self, csv_file): + df = pd.read_csv(csv_file) + accession_to_text = {} + for index, row in df.iterrows(): + accession_to_text[row['VolumeName']] = row["Findings_EN"],row['Impressions_EN'] + return accession_to_text + + + def prepare_samples(self): + samples = [] + # patient_folders = glob.glob(os.path.join(self.data_folder, '*')) + + # Read labels once outside the loop + import json + + with open(self.json_file, 'r') as f: + items = [json.loads(line) for line in f] + # breakpoint() + # 2. 提取所有 volume_path + volume_paths = [item['volume_path'] + for item in items + if 'volume_path' in item] + + disease_mask_paths = [item['disease_mask'] + for item in items + if 'disease_mask' in item] + + organ_mask_paths = [item['organ_mask'] + for item in items + if 'organ_mask' in item] + + disease_findings_list = [item['disease_findings'] + for item in items + if 'disease_findings' in item] + disease_mask_channels = [item['disease_mask_channel'] + for item in items + if 'disease_mask_channel' in item] + disease_labels = [item['disease_label'] + for item in items + if 'disease_label' in item] + disease_classes = [item['disease_class'] + for item in items + if 'disease_class' in item] + # breakpoint() + for (ct_file, organ_mask_file, disease_mask_file, disease_findings, disease_mask_channel, disease_label, disease_class) in tqdm.tqdm(zip(volume_paths, organ_mask_paths, disease_mask_paths, disease_findings_list, disease_mask_channels, disease_labels, disease_classes)): + + accession_number = organ_mask_file.split("/")[-1] + + ct_file = '/sd/shuhan/CT-RATE/'+ct_file + organ_file = '/sd/shuhan/CT-RATE/'+organ_mask_file + disease_file = '/sd/shuhan/CT-RATE/'+disease_mask_file + + text_final = disease_findings + + + + samples.append((ct_file, organ_file, disease_file, text_final, disease_mask_channel, disease_label, disease_class)) + + samples = merge_list_elements(samples) + # breakpoint() + return samples + + def __len__(self): + return len(self.samples) + + def nii_img_to_tensor(self, ct_file, organ_file, disease_file, disease_mask_channel, disease_label, disease_class, transform): + if disease_file == None: + + nii_img = nib.load(str(ct_file)) + img_data = nii_img.get_fdata() + + df = pd.read_csv("/sd/shuhan/CT-RATE/metadata/all_metadata.csv") #select the metadata + file_name = ct_file.split("/")[-1] + row = df[df['VolumeName'] == file_name] + slope = float(row["RescaleSlope"].iloc[0]) + intercept = float(row["RescaleIntercept"].iloc[0]) + xy_spacing = float(row["XYSpacing"].iloc[0][1:][:-2].split(",")[0]) + z_spacing = float(row["ZSpacing"].iloc[0]) + + nii_seg = nib.load(str(organ_file)) + mask_data = nii_seg.get_fdata() + + current = (z_spacing, xy_spacing, xy_spacing) + + img_data = img_data.transpose(2, 0, 1) + tensor = torch.tensor(img_data) + tensor = tensor.unsqueeze(0).unsqueeze(0) + img_data, target_spacing = resize_array(tensor, current) + img_data = img_data[0][0] + + mask_data = mask_data.transpose(2, 0, 1) + tensor = torch.tensor(mask_data) + tensor = tensor.unsqueeze(0).unsqueeze(0) + mask_data = resize_mask(tensor, current) + mask_data = mask_data[0][0] + # breakpoint() + assert mask_data.shape == img_data.shape + fg_mask = (mask_data>0).astype(np.uint8) + mask_data = (((mask_data ) / 300)).astype(np.float32) * 2 -1 + + hu_min, hu_max = -1000, 1000 + img_data = np.clip(img_data, hu_min, hu_max) + + bg_np = np.ones_like(img_data) * -1000 + + img_data = img_data*fg_mask + bg_np*(1-fg_mask) + + + img_data = (((img_data ) / 1000)).astype(np.float32) + start_id = np.random.randint(0, img_data.shape[0]-self.sample_length) + # img_data = img_data[start_id:start_id+self.sample_length] + # mask_data = mask_data[start_id:start_id+self.sample_length] + + # slices=[] + img_data = img_data/2.0 + mask_data = mask_data/2.0 + + img_data = torch.tensor(img_data) + mask_data = torch.tensor(mask_data) + + + img_data = img_data.unsqueeze(0) + mask_data = mask_data.unsqueeze(0) + + return img_data, mask_data, target_spacing, file_name + else: + ct_nii = nib.load(str(ct_file)) + img_data = ct_nii.get_fdata() + + df = pd.read_csv("/sd/shuhan/CT-RATE/metadata/all_metadata.csv") #select the metadata + file_name = ct_file.split("/")[-1] + row = df[df['VolumeName'] == file_name] + slope = float(row["RescaleSlope"].iloc[0]) + intercept = float(row["RescaleIntercept"].iloc[0]) + xy_spacing = float(row["XYSpacing"].iloc[0][1:][:-2].split(",")[0]) + z_spacing = float(row["ZSpacing"].iloc[0]) + + current = (z_spacing, xy_spacing, xy_spacing) + img_data = img_data.transpose(2, 0, 1) + tensor = torch.tensor(img_data) + tensor = tensor.unsqueeze(0).unsqueeze(0) + img_data, target_spacing = resize_array(tensor, current) + img_data = img_data[0][0] + + organ_nii = nib.load(str(organ_file)) + organ_mask = organ_nii.get_fdata() + organ_mask = organ_mask.transpose(2, 0, 1) + tensor = torch.tensor(organ_mask) + tensor = tensor.unsqueeze(0).unsqueeze(0) + organ_mask = resize_mask(tensor, current) + organ_mask = organ_mask[0][0] + + # disease_nii = nib.load(str(disease_file)) + # disease_mask = disease_nii.get_fdata()[int(disease_mask_channel)] + disease_mask = build_multiclass_mask(disease_file,disease_mask_channel,disease_class) + disease_mask = disease_mask.transpose(2, 0, 1) + tensor = torch.tensor(disease_mask) + tensor = tensor.unsqueeze(0).unsqueeze(0) + disease_mask = resize_mask(tensor, current) + disease_mask = disease_mask[0][0] + + assert organ_mask.shape == img_data.shape + + num_slices = img_data.shape[0] + + # 找到 mask>0 的切片索引 + valid_indices = np.where(disease_mask.sum(axis=(1, 2)) > 0)[0] + + if len(valid_indices) > 0: + # 只保留能完整取 sample_length 的起点 + valid_start_ids = valid_indices[valid_indices <= num_slices - self.sample_length] + + if len(valid_start_ids) > 0: + start_id = np.random.choice(valid_start_ids) + else: + # 如果没有合法起点就退化到随机,但保证能取 sample_length + start_id = np.random.randint(0, num_slices - self.sample_length + 1) + else: + # 如果 mask 全部为0,则完全随机 + start_id = np.random.randint(0, num_slices - self.sample_length + 1) + + end_id = start_id + self.sample_length # 保证长度严格等于 sample_length + # img_data = img_data[start_id:end_id] + # organ_mask = organ_mask[start_id:end_id] + # disease_mask = disease_mask[start_id:end_id] + # img_data = img_data[start_id] + # organ_mask = organ_mask[start_id] + # disease_mask = disease_mask[start_id] + + organ_disease_mask = copy.deepcopy(organ_mask) + # organ_disease_mask[disease_mask>0] = disease_class+100 + organ_disease_mask[disease_mask!=0] = disease_mask[disease_mask != 0]+100 + + organ_mask = (((organ_mask ) / 300)).astype(np.float32) * 2 -1 + organ_disease_mask = (((organ_disease_mask ) / 300)).astype(np.float32) * 2 -1 + + hu_min, hu_max = -1000, 1000 + img_data = np.clip(img_data, hu_min, hu_max) + img_data = (((img_data ) / 1000)).astype(np.float32) + + img_data = torch.tensor(img_data) + organ_mask = torch.tensor(organ_mask) + organ_disease_mask = torch.tensor(organ_disease_mask) + + img_data = img_data/2.0 + organ_mask = organ_mask/2.0 + organ_disease_mask = organ_disease_mask/2.0 + + img_data = img_data.unsqueeze(0) + organ_mask = organ_mask.unsqueeze(0) + organ_disease_mask = organ_disease_mask.unsqueeze(0) + + + # img_data=img_data.repeat(3,1,1,1) + # organ_mask=organ_mask.repeat(3,1,1,1) + # organ_disease_mask=organ_disease_mask.repeat(3,1,1,1) + + + # return img_data, organ_mask, organ_disease_mask, target_spacing, file_name + # return img_data, organ_mask, target_spacing, file_name + return img_data, organ_disease_mask, target_spacing, file_name + + + + def __getitem__(self, index): + ct_file, organ_file, disease_file, input_text, disease_mask_channel, disease_label, disease_class = self.samples[index] + video_tensor, volume_seg, spacing, file_name = self.nii_to_tensor(ct_file, organ_file, disease_file, disease_mask_channel, disease_label, disease_class) + input_text = str(input_text) + input_text = input_text.replace('"', '') + input_text = input_text.replace('\'', '') + input_text = input_text.replace('(', '') + input_text = input_text.replace(')', '') + + # return video_tensor, input_text + example = {} + example['name'] = file_name + example['volume_data'] = video_tensor.float() + example['volume_seg'] = volume_seg.float() + example['spacing'] = spacing + example['input_text'] = input_text + example['disease_mask_channel'] = disease_mask_channel + example['disease_label'] = disease_label + return example \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/ct_clip_data_train.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/ct_clip_data_train.py new file mode 100644 index 0000000000000000000000000000000000000000..7b7b6e5bc77383c10c256a130e173ad079239749 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/ct_clip_data_train.py @@ -0,0 +1,432 @@ +import os +import glob +import json +import torch +import pandas as pd +import numpy as np +from PIL import Image +from torch.utils.data import Dataset +import torchvision.transforms as transforms +from functools import partial +import torch.nn.functional as F +import nibabel as nib +import tqdm +import copy +from collections import defaultdict + +def merge_list_elements(data): + """ + data: list of lists,每个元素长度为7 + 返回: 合并后的list,仍然是7列 + """ + merged = defaultdict(lambda: [[] for _ in range(5)]) + + for item in data: + key = (item[0], item[1]) # 前两个元素作为key + for i in range(5): # 后面五列分别合并 + merged[key][i].append(item[2+i]) + + # 重新组装结果 + result = [] + for (a, b), rest in merged.items(): + result.append([a, b] + rest) + return result + + +def build_multiclass_mask( + disease_files, + disease_mask_channels, + disease_class_ids, + *, + channel_axis=0, + threshold=0.5, + out_dtype=np.float32, +): + """ + 将 (file, channel, class_id) 成对列表合并为单通道多类别mask(无重叠假设)。 + 背景=0,其它为对应 class_id。 + + 参数 + ---- + disease_files : List[str] + NIfTI 文件路径列表 + disease_mask_channels : List[int] + 通道索引列表(与 disease_files 一一对应) + disease_class_ids : List[int] + 每个 (file, channel) 对应的最终类别 id + channel_axis : int + 通道所在维度(默认 0) + threshold : float + 二值化阈值;mask > threshold 视为前景 + out_dtype : numpy dtype + 输出标签图的数据类型,默认 uint16 + """ + + if not (len(disease_files) == len(disease_mask_channels) == len(disease_class_ids)): + raise ValueError("disease_files、disease_mask_channels、disease_class_ids 长度必须一致") + + # 用第一份确定空间形状 + first_img = nib.load(str(disease_files[0])) + first_data = first_img.get_fdata(dtype=np.float32) + sample_mask = np.take(first_data, int(disease_mask_channels[0]), axis=channel_axis) + spatial_shape = sample_mask.shape + + label_map = np.zeros(spatial_shape, dtype=out_dtype) + + for fpath, ch, cid in zip(disease_files, disease_mask_channels, disease_class_ids): + img = nib.load(str(fpath)) + data = img.get_fdata(dtype=np.float32) + # breakpoint() + mask = np.take(data, int(ch), axis=channel_axis) + + if mask.shape != spatial_shape: + raise ValueError( + f"{fpath} 的 mask 形状 {mask.shape} 与期望 {spatial_shape} 不一致,请先重采样" + ) + + # 二值化并赋类标 + label_map[mask > threshold] = cid + + return label_map + +def resize_array(array, current_spacing): + """ + Resize the array to match the target spacing. + + Args: + array (torch.Tensor): Input array to be resized. + current_spacing (tuple): Current voxel spacing (z_spacing, xy_spacing, xy_spacing). + target_spacing (tuple): Target voxel spacing (target_z_spacing, target_x_spacing, target_y_spacing). + + Returns: + np.ndarray: Resized array. + """ + # Calculate new dimensions + original_shape = array.shape[2:] + + + new_shape = [original_shape[0], 512, 512] + scaling_factors = [new_shape[i] / original_shape[i] for i in range(len(original_shape))] + resized_spacing = [current_spacing[i] / scaling_factors[i] for i in range(len(original_shape))] + # Resize the array + resized_array = F.interpolate(array, size=new_shape, mode='trilinear', align_corners=False).cpu().numpy() + # breakpoint() + return resized_array, resized_spacing + +def resize_mask(array, current_spacing): + """ + Resize the array to match the target spacing. + + Args: + array (torch.Tensor): Input array to be resized. + current_spacing (tuple): Current voxel spacing (z_spacing, xy_spacing, xy_spacing). + target_spacing (tuple): Target voxel spacing (target_z_spacing, target_x_spacing, target_y_spacing). + + Returns: + np.ndarray: Resized array. + """ + # Calculate new dimensions + original_shape = array.shape[2:] + + new_shape = [original_shape[0], 512, 512] + + resized_array = F.interpolate(array, size=new_shape, mode='nearest').cpu().numpy() + # breakpoint() + return resized_array + +class CTReportDataset(Dataset): + def __init__(self, data_folder, csv_file, min_slices=20, resize_dim=500, force_num_frames=True): + self.data_folder = data_folder + self.min_slices = min_slices + self.accession_to_text = self.load_accession_text(csv_file) + self.paths=[] + self.samples = self.prepare_samples() + percent = 80 + num_files = int((len(self.samples) * percent) / 100) + #num_files = 2286 + self.samples = self.samples[:num_files] + print(len(self.samples)) + self.count = 0 + + + #self.resize_dim = resize_dim + #self.resize_transform = transforms.Resize((resize_dim, resize_dim)) + self.transform = transforms.Compose([ + transforms.Resize((resize_dim,resize_dim)), + transforms.ToTensor() + ]) + self.nii_to_tensor = partial(self.nii_img_to_tensor, transform = self.transform) + self.sample_length=65 + + def load_accession_text(self, csv_file): + df = pd.read_csv(csv_file) + accession_to_text = {} + for index, row in df.iterrows(): + # breakpoint() + accession_to_text[row['VolumeName']] = row["Findings_EN"],row['Impressions_EN'] + + return accession_to_text + + + def prepare_samples(self): + samples = [] + import json + + with open('/sd/qichen/full_ct_gen/CT-RATE/disease_mask_json/disease_train_single_prompt_checked_label.json', 'r') as f: + items = [json.loads(line) for line in f] + + # 2. 提取所有 volume_path + volume_paths = [item['volume_path'] + for item in items + if 'volume_path' in item] + + effusion_mask_paths = [item['disease_mask'] + for item in items + if 'disease_mask' in item] + + organ_mask_paths = [item['organ_mask'] + for item in items + if 'organ_mask' in item] + + disease_findings_list = [item['disease_findings'] + for item in items + if 'disease_findings' in item] + + disease_mask_channels = [item['disease_mask_channel'] + for item in items + if 'disease_mask_channel' in item] + disease_labels = [item['disease_label'] + for item in items + if 'disease_label' in item] + disease_classes = [item['disease_class'] + for item in items + if 'disease_class' in item] + + for (ct_file, organ_mask_file, disease_mask_file, disease_findings, disease_mask_channel, disease_label, disease_class) in tqdm.tqdm(zip(volume_paths, organ_mask_paths, effusion_mask_paths, disease_findings_list, disease_mask_channels, disease_labels, disease_classes)): + + accession_number = organ_mask_file.split("/")[-1] + + ct_file = '/sd/shuhan/CT-RATE/'+ct_file + organ_file = '/sd/shuhan/CT-RATE/'+organ_mask_file + disease_file = '/sd/shuhan/CT-RATE/'+disease_mask_file + + impression_text = disease_findings + + samples.append((ct_file, organ_file, disease_file, impression_text, disease_mask_channel, disease_label, disease_class)) + + samples = merge_list_elements(samples) + + with open('/sd/qichen/full_ct_gen/CT-RATE/train_no_disease_2_checked.json', 'r') as f: + items = [json.loads(line) for line in f] + + # 2. 提取所有 volume_path + volume_paths = [item['volume_path'] + for item in items + if 'volume_path' in item] + organ_masks = [item['organ_mask'] + for item in items + if 'organ_mask' in item] + for nii_file, organ_mask in tqdm.tqdm(zip(volume_paths, organ_masks)): + + accession_number = nii_file.split("/")[-1] + + seg_file = '/sd/shuhan/CT-RATE/'+ organ_mask + nii_file = '/sd/shuhan/CT-RATE/'+nii_file + + if accession_number not in self.accession_to_text: + continue + + impression_text = self.accession_to_text[accession_number] + + if impression_text == "Not given.": + impression_text="" + + input_text_concat = "" + for text in impression_text: + input_text_concat = input_text_concat + str(text) + input_text_concat = impression_text[0] + input_text = f'{impression_text}' + samples.append((nii_file, seg_file, None, input_text_concat, None, None, None)) + + return samples + + def __len__(self): + return len(self.samples) + + + def nii_img_to_tensor(self, ct_file, organ_file, disease_file, disease_mask_channel, disease_label, disease_class, transform): + if disease_file == None: + + nii_img = nib.load(str(ct_file)) + img_data = nii_img.get_fdata() + + df = pd.read_csv("/sd/shuhan/CT-RATE/metadata/all_metadata.csv") #select the metadata + file_name = ct_file.split("/")[-1] + row = df[df['VolumeName'] == file_name] + slope = float(row["RescaleSlope"].iloc[0]) + intercept = float(row["RescaleIntercept"].iloc[0]) + xy_spacing = float(row["XYSpacing"].iloc[0][1:][:-2].split(",")[0]) + z_spacing = float(row["ZSpacing"].iloc[0]) + + nii_seg = nib.load(str(organ_file)) + mask_data = nii_seg.get_fdata() + + current = (z_spacing, xy_spacing, xy_spacing) + + img_data = img_data.transpose(2, 0, 1) + tensor = torch.tensor(img_data) + tensor = tensor.unsqueeze(0).unsqueeze(0) + img_data, target_spacing = resize_array(tensor, current) + img_data = img_data[0][0] + + mask_data = mask_data.transpose(2, 0, 1) + tensor = torch.tensor(mask_data) + tensor = tensor.unsqueeze(0).unsqueeze(0) + mask_data = resize_mask(tensor, current) + mask_data = mask_data[0][0] + # breakpoint() + assert mask_data.shape == img_data.shape + fg_mask = (mask_data>0).astype(np.uint8) + mask_data = (((mask_data ) / 300)).astype(np.float32) * 2 -1 + + hu_min, hu_max = -1000, 1000 + img_data = np.clip(img_data, hu_min, hu_max) + + bg_np = np.ones_like(img_data) * -1000 + + img_data = img_data*fg_mask + bg_np*(1-fg_mask) + + + img_data = (((img_data ) / 1000)).astype(np.float32) + start_id = np.random.randint(0, img_data.shape[0]-self.sample_length) + img_data = img_data[start_id:start_id+self.sample_length] + mask_data = mask_data[start_id:start_id+self.sample_length] + + # slices=[] + img_data = img_data/2.0 + mask_data = mask_data/2.0 + + img_data = torch.tensor(img_data) + mask_data = torch.tensor(mask_data) + + + img_data = img_data.unsqueeze(0) + mask_data = mask_data.unsqueeze(0) + + return img_data, mask_data, target_spacing, file_name + else: + ct_nii = nib.load(str(ct_file)) + img_data = ct_nii.get_fdata() + + df = pd.read_csv("/sd/shuhan/CT-RATE/metadata/all_metadata.csv") #select the metadata + file_name = ct_file.split("/")[-1] + row = df[df['VolumeName'] == file_name] + slope = float(row["RescaleSlope"].iloc[0]) + intercept = float(row["RescaleIntercept"].iloc[0]) + xy_spacing = float(row["XYSpacing"].iloc[0][1:][:-2].split(",")[0]) + z_spacing = float(row["ZSpacing"].iloc[0]) + + current = (z_spacing, xy_spacing, xy_spacing) + img_data = img_data.transpose(2, 0, 1) + tensor = torch.tensor(img_data) + tensor = tensor.unsqueeze(0).unsqueeze(0) + img_data, target_spacing = resize_array(tensor, current) + img_data = img_data[0][0] + + organ_nii = nib.load(str(organ_file)) + organ_mask = organ_nii.get_fdata() + organ_mask = organ_mask.transpose(2, 0, 1) + tensor = torch.tensor(organ_mask) + tensor = tensor.unsqueeze(0).unsqueeze(0) + organ_mask = resize_mask(tensor, current) + organ_mask = organ_mask[0][0] + + # disease_nii = nib.load(str(disease_file)) + disease_mask = build_multiclass_mask(disease_file,disease_mask_channel,disease_class) + # disease_mask = disease_nii.get_fdata()[int(disease_mask_channel)] + disease_mask = disease_mask.transpose(2, 0, 1) + tensor = torch.tensor(disease_mask) + tensor = tensor.unsqueeze(0).unsqueeze(0) + disease_mask = resize_mask(tensor, current) + disease_mask = disease_mask[0][0] + + assert organ_mask.shape == img_data.shape + + num_slices = img_data.shape[0] + + # 找到 mask>0 的切片索引 + valid_indices = np.where(disease_mask.sum(axis=(1, 2)) > 0)[0] + + if len(valid_indices) > 0: + # 只保留能完整取 sample_length 的起点 + valid_start_ids = valid_indices[valid_indices <= num_slices - self.sample_length] + + if len(valid_start_ids) > 0: + start_id = np.random.choice(valid_start_ids) + else: + # 如果没有合法起点就退化到随机,但保证能取 sample_length + start_id = np.random.randint(0, num_slices - self.sample_length + 1) + else: + # 如果 mask 全部为0,则完全随机 + start_id = np.random.randint(0, num_slices - self.sample_length + 1) + + end_id = start_id + self.sample_length # 保证长度严格等于 sample_length + img_data = img_data[start_id:end_id] + organ_mask = organ_mask[start_id:end_id] + disease_mask = disease_mask[start_id:end_id] + # img_data = img_data[start_id] + # organ_mask = organ_mask[start_id] + # disease_mask = disease_mask[start_id] + + organ_disease_mask = copy.deepcopy(organ_mask) + # organ_disease_mask[disease_mask>0] = disease_class+100 + organ_disease_mask[disease_mask!=0] = disease_mask[disease_mask != 0]+100 + + organ_mask = (((organ_mask ) / 300)).astype(np.float32) * 2 -1 + organ_disease_mask = (((organ_disease_mask ) / 300)).astype(np.float32) * 2 -1 + + hu_min, hu_max = -1000, 1000 + img_data = np.clip(img_data, hu_min, hu_max) + img_data = (((img_data ) / 1000)).astype(np.float32) + + img_data = torch.tensor(img_data) + organ_mask = torch.tensor(organ_mask) + organ_disease_mask = torch.tensor(organ_disease_mask) + + img_data = img_data/2.0 + organ_mask = organ_mask/2.0 + organ_disease_mask = organ_disease_mask/2.0 + + img_data = img_data.unsqueeze(0) + organ_mask = organ_mask.unsqueeze(0) + organ_disease_mask = organ_disease_mask.unsqueeze(0) + + + # img_data=img_data.repeat(3,1,1,1) + # organ_mask=organ_mask.repeat(3,1,1,1) + # organ_disease_mask=organ_disease_mask.repeat(3,1,1,1) + + + # return img_data, organ_mask, organ_disease_mask, target_spacing, file_name + # return img_data, organ_mask, target_spacing, file_name + return img_data, organ_disease_mask, target_spacing, file_name + + + + def __getitem__(self, index): + ct_file, organ_file, disease_file, input_text, disease_mask_channel, disease_label, disease_class = self.samples[index] + video_tensor, volume_seg, spacing, file_name = self.nii_to_tensor(ct_file, organ_file, disease_file, disease_mask_channel, disease_label, disease_class) + input_text = str(input_text) + input_text = input_text.replace('"', '') + input_text = input_text.replace('\'', '') + input_text = input_text.replace('(', '') + input_text = input_text.replace(')', '') + + # return video_tensor, input_text + example = {} + example['name'] = file_name + example['volume_data'] = video_tensor.float() + example['volume_seg'] = volume_seg.float() + example['spacing'] = spacing + example['input_text'] = input_text + return example \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/mask_generation.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/mask_generation.py new file mode 100644 index 0000000000000000000000000000000000000000..c8ac71405628b5dd6fdc9ab2c599ead4c260845b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/mask_generation.py @@ -0,0 +1,61 @@ +import os +import numpy as np +from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy + +class volume_base(Dataset): + def __init__(self, + data_root, + data_name, + data_file, + data_repeat=1 + ): + + self.data_root = data_root + planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + file_list = [] + for line in open(data_file): + name = line.strip().split()[1].split('.nii.gz')[0] + name = name.split('/')[-1] + file_list.append(name) + + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self.sample_length=16 + self.sample=False + self._data_repeat = data_repeat + + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + seg_npz = np.load(npz_file)['seg'][0] + + volume_seg = volume_seg.astype(np.float32) + + return volume_seg + +class volume_train(volume_base): + def __init__(self, data_file='data_split/liver/train.txt', **kwargs): + super().__init__(data_file=data_file, **kwargs) + + +class volume_val(volume_base): + def __init__(self, data_file='data_split/liver/eval.txt', **kwargs): + super().__init__(data_file=data_file, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class volume_test(volume_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset.py new file mode 100644 index 0000000000000000000000000000000000000000..e9387be62d79305474c66be20b32ce6b5d8e9014 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset.py @@ -0,0 +1,103 @@ +import os +import numpy as np +# from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy +from batchgenerators.utilities.file_and_folder_operations import join, load_json, isfile, save_json, maybe_mkdir_p + + +class slice_base(Dataset): + def __init__(self, + data_root, + data_name, + data_repeat=1, + planner='nnUNetPlans_3d_fullres', + data_file='data_split/liver/train.txt', + phase='train', + ): + + self.data_root = data_root + # planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + # self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + + file_list = [] + if 'liver' in data_file: + for line in open(data_file): + name = line.strip().split()[1].split('.nii.gz')[0] + name = name.split('/')[-1] + file_list.append(name) + else: + file_list = load_json(data_file)[0][phase] + + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self._data_repeat = data_repeat + + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + # print('data_npz', data_npz.shape) + # print('seg_npz', seg_npz.shape) + # breakpoint() + pos_id = np.random.randint(0, data_npz.shape[0]) + # pos_id = data_npz.shape[0]//2 + slice_data = data_npz[pos_id] + slice_seg = seg_npz[pos_id] + + # slice_data = np.pad(slice_data,((0,0),(8,8),(8,8)),'constant') + # breakpoint() + slice_data[slice_data <= -175] = -175 + slice_data[slice_data >= 250] = 250 + # breakpoint() + slice_data = (slice_data+175)/425.0 + + # tumor_mask = (slice_seg==2).astype(np.float32) + slice_seg = slice_seg.astype(np.float32) + foreground_mask = (slice_seg>0).astype(np.float32) + masked_data = (1-foreground_mask)*slice_data + + slice_data = slice_data[:,:,None].astype(np.float32) + masked_data = masked_data[:,:,None].astype(np.float32) + + # tumor_mask = resize(tumor_mask, (64,64), order=0) + slice_seg = cv2.resize(slice_seg,(64,64),interpolation=cv2.INTER_NEAREST) + slice_seg = slice_seg[:,:,None].astype(np.float32) + # print(np.unique(slice_seg)) + # breakpoint() + if (slice_data.shape[0] != 512) or (slice_data.shape[1] != 512): + print(self.list[i],slice_data.shape,masked_data.shape,slice_seg.shape) + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['slice_data'] = slice_data * 2 - 1 + example['masked_data'] = masked_data * 2 - 1 + example['tumor_mask'] = slice_seg + # breakpoint() + return example + +class slice_train(slice_base): + def __init__(self, phase='train', **kwargs): + super().__init__(phase=phase, **kwargs) + + +class slice_val(slice_base): + def __init__(self, phase='val', **kwargs): + super().__init__(phase=phase, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class slice_test(slice_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset__.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset__.py new file mode 100644 index 0000000000000000000000000000000000000000..2e53d2b753b22444d3415c651efad0771dca662d --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset__.py @@ -0,0 +1,81 @@ +import os +import numpy as np +# from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy + +class slice_base(Dataset): + def __init__(self, + data_root, + data_name, + # data_file, + data_repeat=1 + ): + + self.data_root = data_root + planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + self.list.sort() + # breakpoint() + self.list = self.list[:5] + # file_list = [] + # with open(data_file, 'r') as f: + # file_list = f.readlines() + # file_list = [i.split('\n')[0] for i in file_list] + # self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + # breakpoint() + self._length = len(self.list) + self._data_repeat = data_repeat + + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + + # pos_id = np.random.randint(0, data_npz.shape[0]) + # pos_id = data_npz.shape[0]//2 + # slice_data = data_npz[pos_id] + # slice_seg = seg_npz[pos_id] + + # slice_data = np.pad(slice_data,((0,0),(8,8),(8,8)),'constant') + # breakpoint() + data_npz[data_npz <= -175] = -175 + data_npz[data_npz >= 250] = 250 + # breakpoint() + data_npz = (data_npz+175)/425.0 + + data_npz = data_npz[:,:,:,None].astype(np.float32) + + # breakpoint() + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['slice_data'] = data_npz * 2 - 1 + + # breakpoint() + return example + +class slice_train(slice_base): + def __init__(self, **kwargs): + super().__init__( **kwargs) + + +class slice_val(slice_base): + def __init__(self, **kwargs): + super().__init__(**kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class slice_test(slice_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_28class.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_28class.py new file mode 100644 index 0000000000000000000000000000000000000000..de79231129330e28794129304585ca12e9b1476c --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_28class.py @@ -0,0 +1,99 @@ +import os +import numpy as np +from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy +from batchgenerators.utilities.file_and_folder_operations import load_json + +class slice_base(Dataset): + def __init__(self, + data_root, + data_name, + data_file, + data_repeat=1, + phase='train', + planner='nnUNetPlans_3d_fullres' + ): + + self.data_root = data_root + # planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + # self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + + data_splits = load_json(data_file) + file_list = data_splits[0][phase] + # breakpoint() + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self._data_repeat = data_repeat + + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + # print(npz_file, data_npz.shape, seg_npz.shape) + # breakpoint() + pos_id = np.random.randint(0, data_npz.shape[0]) + # pos_id = data_npz.shape[0]//2 + slice_data = data_npz[pos_id] + slice_seg = seg_npz[pos_id] + + # slice_data = np.pad(slice_data,((0,0),(8,8),(8,8)),'constant') + # breakpoint() + slice_data[slice_data <= -175] = -175 + slice_data[slice_data >= 250] = 250 + # breakpoint() + slice_data = (slice_data+175)/425.0 + + + # slice_data = resize(slice_data, (512,512)) + # slice_seg = resize(slice_seg.astype(np.float32), (512,512), order=0, anti_aliasing=False) + # breakpoint() + + # tumor_mask = (slice_seg==2).astype(np.float32) + slice_seg = slice_seg.astype(np.float32) + foreground_mask = (slice_seg>0).astype(np.float32) + masked_data = (1-foreground_mask)*slice_data + + slice_data = slice_data[:,:,None].astype(np.float32) + masked_data = masked_data[:,:,None].astype(np.float32) + + # breakpoint() + # tumor_mask = resize(tumor_mask, (64,64), order=0) + slice_seg = cv2.resize(slice_seg,(64,64),interpolation=cv2.INTER_NEAREST) + slice_seg = slice_seg[:,:,None].astype(np.float32) + # print(np.unique(slice_seg)) + # breakpoint() + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['slice_data'] = slice_data * 2 - 1 + example['masked_data'] = masked_data * 2 - 1 + example['tumor_mask'] = slice_seg + # breakpoint() + return example + +class slice_train(slice_base): + def __init__(self, data_file='data_split/splits_final2.json', phase='train', **kwargs): + super().__init__(data_file=data_file, **kwargs) + + +class slice_val(slice_base): + def __init__(self, data_file='data_split/splits_final2.json', phase='val', **kwargs): + super().__init__(data_file=data_file,phase=phase, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class slice_test(slice_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_28class_randomclass.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_28class_randomclass.py new file mode 100644 index 0000000000000000000000000000000000000000..7d2095b5a685e098685ee2028c817634cf28feb4 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_28class_randomclass.py @@ -0,0 +1,127 @@ +import os +import numpy as np +from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy +# from batchgenerators.utilities.file_and_folder_operations import load_json + +import random +import itertools + +def get_random_classes_from_mask(mask): + # 分析 mask 中有哪些类别 + unique_classes = np.unique(mask) + unique_classes = unique_classes[unique_classes != 0] # 去除背景类(假设背景类为0) + + # 获取所有可能的类别组合 + all_combinations = [] + for i in range(1, len(unique_classes) + 1): + all_combinations.extend(itertools.combinations(unique_classes, i)) + + # 随机选择一个组合 + selected_combination = random.choice(all_combinations) + return selected_combination + +def get_filtered_mask(mask, selected_classes): + # 创建只包含选定类别的 mask + filtered_mask = np.isin(mask, selected_classes) * mask + return filtered_mask + +class slice_base(Dataset): + def __init__(self, + data_root, + data_name, + data_repeat=1, + phase='train', + planner='nnUNetPlans_3d_fullres', + data_file=None + ): + + self.data_root = data_root + # planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + # self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + + data_splits = load_json(data_file) + file_list = data_splits[0][phase] + # breakpoint() + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self._data_repeat = data_repeat + self.class_num = 28 + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + # print(npz_file, data_npz.shape, seg_npz.shape) + # breakpoint() + pos_id = np.random.randint(0, data_npz.shape[0]) + slice_data = data_npz[pos_id] + slice_seg = seg_npz[pos_id] + + # breakpoint() + slice_data[slice_data <= -175] = -175 + slice_data[slice_data >= 250] = 250 + slice_data = (slice_data+175)/425.0 + + # print(np.unique(slice_seg)) + if len(np.unique(slice_seg)) > 1: + selected_classes = get_random_classes_from_mask(slice_seg) + slice_seg = get_filtered_mask(slice_seg, selected_classes) + # print(np.unique(slice_seg)) + + # breakpoint() + + # slice_data = resize(slice_data, (512,512)) + # slice_seg = resize(slice_seg.astype(np.float32), (512,512), order=0, anti_aliasing=False) + # breakpoint() + + # tumor_mask = (slice_seg==2).astype(np.float32) + slice_seg = slice_seg.astype(np.float32) + foreground_mask = (slice_seg>0).astype(np.float32) + masked_data = (1-foreground_mask)*slice_data + + slice_data = slice_data[:,:,None].astype(np.float32) + masked_data = masked_data[:,:,None].astype(np.float32) + + # breakpoint() + # tumor_mask = resize(tumor_mask, (64,64), order=0) + slice_seg = cv2.resize(slice_seg,(64,64),interpolation=cv2.INTER_NEAREST) + slice_seg = slice_seg[:,:,None].astype(np.float32) + if (slice_data.shape[0] != 512) or (slice_data.shape[1] != 512): + print(self.list[i],slice_data.shape,masked_data.shape,slice_seg.shape) + # print(np.unique(slice_seg)) + # breakpoint() + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['slice_data'] = slice_data * 2 - 1 + example['masked_data'] = masked_data * 2 - 1 + example['tumor_mask'] = slice_seg + # breakpoint() + return example + +class slice_train(slice_base): + def __init__(self, phase='train', **kwargs): + super().__init__(**kwargs) + + +class slice_val(slice_base): + def __init__(self, phase='val', **kwargs): + super().__init__(phase=phase, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class slice_test(slice_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_28class_randomclass_kl4.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_28class_randomclass_kl4.py new file mode 100644 index 0000000000000000000000000000000000000000..433c1f35318ec318496c4397c1b6b3bdb1c73831 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_28class_randomclass_kl4.py @@ -0,0 +1,127 @@ +import os +import numpy as np +from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy +from batchgenerators.utilities.file_and_folder_operations import load_json + +import random +import itertools + +def get_random_classes_from_mask(mask): + # 分析 mask 中有哪些类别 + unique_classes = np.unique(mask) + unique_classes = unique_classes[unique_classes != 0] # 去除背景类(假设背景类为0) + + # 获取所有可能的类别组合 + all_combinations = [] + for i in range(1, len(unique_classes) + 1): + all_combinations.extend(itertools.combinations(unique_classes, i)) + + # 随机选择一个组合 + selected_combination = random.choice(all_combinations) + return selected_combination + +def get_filtered_mask(mask, selected_classes): + # 创建只包含选定类别的 mask + filtered_mask = np.isin(mask, selected_classes) * mask + return filtered_mask + +class slice_base(Dataset): + def __init__(self, + data_root, + data_name, + data_repeat=1, + phase='train', + planner='nnUNetPlans_3d_fullres', + data_file=None + ): + + self.data_root = data_root + # planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + # self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + + data_splits = load_json(data_file) + file_list = data_splits[0][phase] + # breakpoint() + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self._data_repeat = data_repeat + self.class_num = 28 + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + # print(npz_file, data_npz.shape, seg_npz.shape) + # breakpoint() + pos_id = np.random.randint(0, data_npz.shape[0]) + slice_data = data_npz[pos_id] + slice_seg = seg_npz[pos_id] + + # breakpoint() + slice_data[slice_data <= -175] = -175 + slice_data[slice_data >= 250] = 250 + slice_data = (slice_data+175)/425.0 + + # print(np.unique(slice_seg)) + if len(np.unique(slice_seg)) > 1: + selected_classes = get_random_classes_from_mask(slice_seg) + slice_seg = get_filtered_mask(slice_seg, selected_classes) + # print(np.unique(slice_seg)) + + # breakpoint() + + # slice_data = resize(slice_data, (512,512)) + # slice_seg = resize(slice_seg.astype(np.float32), (512,512), order=0, anti_aliasing=False) + # breakpoint() + + # tumor_mask = (slice_seg==2).astype(np.float32) + slice_seg = slice_seg.astype(np.float32) + foreground_mask = (slice_seg>0).astype(np.float32) + masked_data = (1-foreground_mask)*slice_data + + slice_data = slice_data[:,:,None].astype(np.float32) + masked_data = masked_data[:,:,None].astype(np.float32) + + # breakpoint() + # tumor_mask = resize(tumor_mask, (64,64), order=0) + slice_seg = cv2.resize(slice_seg,(128,128),interpolation=cv2.INTER_NEAREST) + slice_seg = slice_seg[:,:,None].astype(np.float32) + if (slice_data.shape[0] != 512) or (slice_data.shape[1] != 512): + print(self.list[i],slice_data.shape,masked_data.shape,slice_seg.shape) + # print(np.unique(slice_seg)) + # breakpoint() + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['slice_data'] = slice_data * 2 - 1 + example['masked_data'] = masked_data * 2 - 1 + example['tumor_mask'] = slice_seg + # breakpoint() + return example + +class slice_train(slice_base): + def __init__(self, phase='train', **kwargs): + super().__init__(**kwargs) + + +class slice_val(slice_base): + def __init__(self, phase='val', **kwargs): + super().__init__(phase=phase, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class slice_test(slice_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_resize.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_resize.py new file mode 100644 index 0000000000000000000000000000000000000000..c6a1947ac15fb14f791800d0746a7ae1e796e429 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_abdomenatlas_resize.py @@ -0,0 +1,97 @@ +import os +import numpy as np +from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy +from batchgenerators.utilities.file_and_folder_operations import load_json + +class slice_base(Dataset): + def __init__(self, + data_root, + data_name, + data_file, + data_repeat=1, + phase='train' + ): + + self.data_root = data_root + planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + # self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + + data_splits = load_json(data_file) + file_list = data_splits[0][phase] + # breakpoint() + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self._data_repeat = data_repeat + + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + # breakpoint() + pos_id = np.random.randint(0, data_npz.shape[0]) + # pos_id = data_npz.shape[0]//2 + slice_data = data_npz[pos_id] + slice_seg = seg_npz[pos_id] + + # slice_data = np.pad(slice_data,((0,0),(8,8),(8,8)),'constant') + # breakpoint() + slice_data[slice_data <= -175] = -175 + slice_data[slice_data >= 250] = 250 + # breakpoint() + slice_data = (slice_data+175)/425.0 + + + slice_data = resize(slice_data, (512,512)) + slice_seg = resize(slice_seg.astype(np.float32), (512,512), order=0, anti_aliasing=False) + # breakpoint() + + # tumor_mask = (slice_seg==2).astype(np.float32) + slice_seg = slice_seg.astype(np.float32) + foreground_mask = (slice_seg>0).astype(np.float32) + masked_data = (1-foreground_mask)*slice_data + + slice_data = slice_data[:,:,None].astype(np.float32) + masked_data = masked_data[:,:,None].astype(np.float32) + + # breakpoint() + # tumor_mask = resize(tumor_mask, (64,64), order=0) + slice_seg = cv2.resize(slice_seg,(64,64),interpolation=cv2.INTER_NEAREST) + slice_seg = slice_seg[:,:,None].astype(np.float32) + # print(np.unique(slice_seg)) + # breakpoint() + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['slice_data'] = slice_data * 2 - 1 + example['masked_data'] = masked_data * 2 - 1 + example['tumor_mask'] = slice_seg + # breakpoint() + return example + +class slice_train(slice_base): + def __init__(self, data_file='data_split/splits_final.json', phase='train', **kwargs): + super().__init__(data_file=data_file, **kwargs) + + +class slice_val(slice_base): + def __init__(self, data_file='data_split/splits_final.json', phase='val', **kwargs): + super().__init__(data_file=data_file,phase=phase, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class slice_test(slice_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_infer.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_infer.py new file mode 100644 index 0000000000000000000000000000000000000000..4416966f0e6355055edeb71164fb956153aaae14 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_infer.py @@ -0,0 +1,94 @@ +import os +import numpy as np +# from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy + +class slice_base(Dataset): + def __init__(self, + data_root, + data_name, + data_file, + data_repeat=1 + ): + + self.data_root = data_root + planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + # self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + file_list = [] + for line in open(data_file): + name = line.strip().split()[1].split('.nii.gz')[0] + name = name.split('/')[-1] + file_list.append(name) + # breakpoint() + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self._data_repeat = data_repeat + + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + # breakpoint() + # pos_id = np.random.randint(0, data_npz.shape[0]) + # pos_id = data_npz.shape[0]//2 + pos_id_list = np.where(np.any(data_npz==2, axis=(1, 2)))[0] + pos_id = pos_id_list[len(pos_id_list)//2] + # breakpoint() + slice_data = data_npz[pos_id] + slice_seg = seg_npz[pos_id] + # breakpoint() + # slice_data = np.pad(slice_data,((0,0),(8,8),(8,8)),'constant') + # breakpoint() + slice_data[slice_data <= -175] = -175 + slice_data[slice_data >= 250] = 250 + # breakpoint() + slice_data = (slice_data+175)/425.0 + + # tumor_mask = (slice_seg==2).astype(np.float32) + slice_seg = slice_seg.astype(np.float32) + foreground_mask = (slice_seg>0).astype(np.float32) + masked_data = (1-foreground_mask)*slice_data + + slice_data = slice_data[:,:,None].astype(np.float32) + masked_data = masked_data[:,:,None].astype(np.float32) + + # tumor_mask = resize(tumor_mask, (64,64), order=0) + slice_seg = cv2.resize(slice_seg,(64,64),interpolation=cv2.INTER_NEAREST) + slice_seg = slice_seg[:,:,None].astype(np.float32) + # print(np.unique(slice_seg)) + # breakpoint() + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['slice_data'] = slice_data * 2 - 1 + example['masked_data'] = masked_data * 2 - 1 + example['tumor_mask'] = slice_seg + # breakpoint() + return example + +class slice_train(slice_base): + def __init__(self, data_file='data_split/liver/train.txt', **kwargs): + super().__init__(data_file=data_file, **kwargs) + + +class slice_val(slice_base): + def __init__(self, data_file='data_split/liver/eval.txt', **kwargs): + super().__init__(data_file=data_file, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class slice_test(slice_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_infer_28class_randomclass.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_infer_28class_randomclass.py new file mode 100644 index 0000000000000000000000000000000000000000..85a7a9bc7054e71e59c62a25e6f3fdeb31c1b2bf --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/slice_dataset_infer_28class_randomclass.py @@ -0,0 +1,99 @@ +import os +import numpy as np +# from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy + +def get_filtered_mask(mask, selected_classes): + # 创建只包含选定类别的 mask + filtered_mask = np.isin(mask, selected_classes) * mask + return filtered_mask + +class slice_base(Dataset): + def __init__(self, + data_root, + data_name, + data_file, + data_repeat=1 + ): + + self.data_root = data_root + planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + # self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + file_list = [] + for line in open(data_file): + name = line.strip().split()[1].split('.nii.gz')[0] + name = name.split('/')[-1] + file_list.append(name) + # breakpoint() + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self._data_repeat = data_repeat + + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + # breakpoint() + # pos_id = np.random.randint(0, data_npz.shape[0]) + # pos_id = data_npz.shape[0]//2 + pos_id_list = np.where(np.any(data_npz==2, axis=(1, 2)))[0] + pos_id = pos_id_list[len(pos_id_list)//2] + # breakpoint() + slice_data = data_npz[pos_id] + slice_seg = seg_npz[pos_id] + + slice_data[slice_data <= -175] = -175 + slice_data[slice_data >= 250] = 250 + slice_data = (slice_data+175)/425.0 + + slice_seg[slice_seg==1] = 5 + slice_seg[slice_seg==2] = 26 + + # tumor_mask = (slice_seg==2).astype(np.float32) + slice_seg = slice_seg.astype(np.float32) + foreground_mask = (slice_seg>0).astype(np.float32) + masked_data = (1-foreground_mask)*slice_data + + slice_data = slice_data[:,:,None].astype(np.float32) + masked_data = masked_data[:,:,None].astype(np.float32) + + # tumor_mask = resize(tumor_mask, (64,64), order=0) + slice_seg = cv2.resize(slice_seg,(64,64),interpolation=cv2.INTER_NEAREST) + slice_seg = slice_seg[:,:,None].astype(np.float32) + # print(np.unique(slice_seg)) + # breakpoint() + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['slice_data'] = slice_data * 2 - 1 + example['masked_data'] = masked_data * 2 - 1 + example['tumor_mask'] = slice_seg + # breakpoint() + return example + +class slice_train(slice_base): + def __init__(self, data_file='data_split/liver/train.txt', **kwargs): + super().__init__(data_file=data_file, **kwargs) + + +class slice_val(slice_base): + def __init__(self, data_file='data_split/liver/eval.txt', **kwargs): + super().__init__(data_file=data_file, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class slice_test(slice_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset copy.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset copy.py new file mode 100644 index 0000000000000000000000000000000000000000..9c011677e051b1ea7ac4c555c456c1bd979232fa --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset copy.py @@ -0,0 +1,83 @@ +import os +import numpy as np +from skimage.transform import resize +from torch.utils.data import Dataset + + +class volume_base(Dataset): + def __init__(self, + data_root, + data_name, + sample=True, + sample_length=16, + data_repeat=1 + ): + + self.data_root = data_root + self.data_name = data_name + self.sample = sample + self.sample_length = sample_length + + data_root = os.path.join(data_root, data_name + '/nnUNetPlans_3d_fullres') + self.list = [os.path.join(data_name + '/nnUNetPlans_3d_fullres',f) for f in os.listdir(data_root) if f.endswith('.npz')] + self._length = len(self.list) + self._data_repeat = data_repeat + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'] + seg_npz = np.load(npz_file)['seg'] + + volume_data = data_npz[0, :, :, :] + volume_seg = seg_npz[0, :, :, :] + + volume_data = (volume_data - np.min(data_npz)) / (np.max(data_npz)-np.min(data_npz)) + if 'Abdomen' in self.data_name: + max_seg = 4 + elif 'Brain' in self.data_name: + max_seg = 3 + volume_seg = volume_seg / max_seg + + volume_data = resize(volume_data, (volume_data.shape[0],512,512)) + volume_seg = resize(volume_seg, (volume_seg.shape[0],512,512), order=0, anti_aliasing=False) + + volume_data = volume_data[:, :, :, None].astype(np.float32) + volume_seg = volume_seg[:, :, :, None].astype(np.float32) + + if self.sample: + start_id = np.random.randint(0, data_npz.shape[1]-self.sample_length) + volume_data = volume_data[start_id:start_id+self.sample_length] + volume_seg = volume_seg[start_id:start_id+self.sample_length] + + rand_idx = np.random.randint(0, volume_data.shape[0]) + volume_ref = volume_data[rand_idx:rand_idx+1] + + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + example['volume_data'] = volume_data * 2 - 1 + example['volume_seg'] = volume_seg * 2 - 1 + example['volume_ref'] = volume_ref * 2 - 1 + + return example + +class volume_train(volume_base): + def __init__(self, **kwargs): + super().__init__(**kwargs) + + +class volume_val(volume_base): + def __init__(self, **kwargs): + super().__init__(**kwargs) + + def __len__(self): + return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class volume_test(volume_base): + def __init__(self, **kwargs): + super().__init__(sample=False, **kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset.py new file mode 100644 index 0000000000000000000000000000000000000000..e9362a689c518d4716dbbdae17c6b18f06e7b2d6 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset.py @@ -0,0 +1,98 @@ +import os +import numpy as np +from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy +from batchgenerators.utilities.file_and_folder_operations import join, load_json, isfile, save_json, maybe_mkdir_p + +class volume_base(Dataset): + def __init__(self, + data_root, + data_name, + data_file, + data_repeat=1, + planner='nnUNetPlans_3d_fullres', + phase='train', + ): + + self.data_root = data_root + # planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + # self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + file_list = [] + if 'liver' in data_file: + for line in open(data_file): + name = line.strip().split()[1].split('.nii.gz')[0] + name = name.split('/')[-1] + file_list.append(name) + else: + file_list = load_json(data_file)[0][phase] + # breakpoint() + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self.sample_length=16 + self.sample=True + self._data_repeat = data_repeat + + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + + # breakpoint() + + if self.sample: + start_id = np.random.randint(0, data_npz.shape[0]-self.sample_length) + volume_data = data_npz[start_id:start_id+self.sample_length] + volume_seg = seg_npz[start_id:start_id+self.sample_length] + + volume_data[volume_data <= -175] = -175 + volume_data[volume_data >= 250] = 250 + volume_data = (volume_data+175)/425.0 + + + volume_seg = volume_seg.astype(np.float32) + foreground_mask = (volume_seg>0).astype(np.float32) + masked_data = (1-foreground_mask)*volume_data + + volume_data = volume_data[:,:,:,None].astype(np.float32) + masked_data = masked_data[:,:,:,None].astype(np.float32) + + # breakpoint() + volume_seg = resize(volume_seg, (volume_seg.shape[0],64,64), order=0, anti_aliasing=False) + # print(np.unique(resize(volume_seg, (volume_seg.shape[0],64,64), order=0, anti_aliasing=False))) + volume_seg = volume_seg[:,:,:,None].astype(np.float32) + + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['volume_data'] = volume_data * 2 - 1 + example['masked_data'] = masked_data * 2 - 1 + example['tumor_mask'] = volume_seg + # breakpoint() + return example + +class volume_train(volume_base): + def __init__(self, phase='train', **kwargs): + super().__init__(phase=phase, **kwargs) + + +class volume_val(volume_base): + def __init__(self, phase='val', **kwargs): + super().__init__(phase=phase, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class volume_test(volume_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_13class_randomclass.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_13class_randomclass.py new file mode 100644 index 0000000000000000000000000000000000000000..12c02a0588538a0a4376761ef44f9d6c059329fa --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_13class_randomclass.py @@ -0,0 +1,122 @@ +import os +import numpy as np +from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy +import random +import itertools +from batchgenerators.utilities.file_and_folder_operations import load_json + +def get_random_classes_from_mask(mask): + # 分析 mask 中有哪些类别 + unique_classes = np.unique(mask) + unique_classes = unique_classes[unique_classes != 0] # 去除背景类(假设背景类为0) + + # 获取所有可能的类别组合 + all_combinations = [] + for i in range(1, len(unique_classes) + 1): + all_combinations.extend(itertools.combinations(unique_classes, i)) + + # 随机选择一个组合 + selected_combination = random.choice(all_combinations) + return selected_combination + +def get_filtered_mask(mask, selected_classes): + # 创建只包含选定类别的 mask + filtered_mask = np.isin(mask, selected_classes) * mask + return filtered_mask + +class volume_base(Dataset): + def __init__(self, + data_root, + data_name, + data_file, + data_repeat=1, + phase='train' + ): + + self.data_root = data_root + planner = 'nnUNetPlans_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + # self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + + data_splits = load_json(data_file) + file_list = data_splits[0][phase] + # breakpoint() + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self.class_num = 13 + + self.sample_length=16 + self.sample=True + self._data_repeat = data_repeat + + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + + # breakpoint() + + if self.sample: + start_id = np.random.randint(0, data_npz.shape[0]-self.sample_length) + volume_data = data_npz[start_id:start_id+self.sample_length] + volume_seg = seg_npz[start_id:start_id+self.sample_length] + + # breakpoint() + + + volume_data[volume_data <= -175] = -175 + volume_data[volume_data >= 250] = 250 + volume_data = (volume_data+175)/425.0 + + if len(np.unique(volume_seg)) > 1: + selected_classes = get_random_classes_from_mask(volume_seg) + volume_seg = get_filtered_mask(volume_seg, selected_classes) + + + volume_seg = volume_seg.astype(np.float32) + foreground_mask = (volume_seg>0).astype(np.float32) + masked_data = (1-foreground_mask)*volume_data + + volume_data = volume_data[:,:,:,None].astype(np.float32) + masked_data = masked_data[:,:,:,None].astype(np.float32) + + # breakpoint() + volume_seg = resize(volume_seg, (volume_seg.shape[0],64,64), order=0, anti_aliasing=False) + # print(np.unique(resize(volume_seg, (volume_seg.shape[0],64,64), order=0, anti_aliasing=False))) + volume_seg = volume_seg[:,:,:,None].astype(np.float32) + + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['volume_data'] = volume_data * 2 - 1 + example['masked_data'] = masked_data * 2 - 1 + example['tumor_mask'] = volume_seg + # breakpoint() + return example + +class volume_train(volume_base): + def __init__(self, phase='train', **kwargs): + super().__init__( phase=phase, **kwargs) + + +class volume_val(volume_base): + def __init__(self, phase='val', **kwargs): + super().__init__(phase=phase, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class volume_test(volume_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_28class_randomclass.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_28class_randomclass.py new file mode 100644 index 0000000000000000000000000000000000000000..16224ea7b6d4d164626aa478f51788dbdcf32210 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_28class_randomclass.py @@ -0,0 +1,122 @@ +import os +import numpy as np +from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy +import random +import itertools +from batchgenerators.utilities.file_and_folder_operations import load_json + +def get_random_classes_from_mask(mask): + # 分析 mask 中有哪些类别 + unique_classes = np.unique(mask) + unique_classes = unique_classes[unique_classes != 0] # 去除背景类(假设背景类为0) + + # 获取所有可能的类别组合 + all_combinations = [] + for i in range(1, len(unique_classes) + 1): + all_combinations.extend(itertools.combinations(unique_classes, i)) + + # 随机选择一个组合 + selected_combination = random.choice(all_combinations) + return selected_combination + +def get_filtered_mask(mask, selected_classes): + # 创建只包含选定类别的 mask + filtered_mask = np.isin(mask, selected_classes) * mask + return filtered_mask + +class volume_base(Dataset): + def __init__(self, + data_root, + data_name, + data_file, + data_repeat=1, + phase='train' + ): + + self.data_root = data_root + planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + # self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + + data_splits = load_json(data_file) + file_list = data_splits[0][phase] + # breakpoint() + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self.class_num = 28 + + self.sample_length=16 + self.sample=True + self._data_repeat = data_repeat + + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + + # breakpoint() + + if self.sample: + start_id = np.random.randint(0, data_npz.shape[0]-self.sample_length) + volume_data = data_npz[start_id:start_id+self.sample_length] + volume_seg = seg_npz[start_id:start_id+self.sample_length] + + # breakpoint() + + + volume_data[volume_data <= -175] = -175 + volume_data[volume_data >= 250] = 250 + volume_data = (volume_data+175)/425.0 + + if len(np.unique(volume_seg)) > 1: + selected_classes = get_random_classes_from_mask(volume_seg) + volume_seg = get_filtered_mask(volume_seg, selected_classes) + + + volume_seg = volume_seg.astype(np.float32) + foreground_mask = (volume_seg>0).astype(np.float32) + masked_data = (1-foreground_mask)*volume_data + + volume_data = volume_data[:,:,:,None].astype(np.float32) + masked_data = masked_data[:,:,:,None].astype(np.float32) + + # breakpoint() + volume_seg = resize(volume_seg, (volume_seg.shape[0],64,64), order=0, anti_aliasing=False) + # print(np.unique(resize(volume_seg, (volume_seg.shape[0],64,64), order=0, anti_aliasing=False))) + volume_seg = volume_seg[:,:,:,None].astype(np.float32) + + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['volume_data'] = volume_data * 2 - 1 + example['masked_data'] = masked_data * 2 - 1 + example['tumor_mask'] = volume_seg + # breakpoint() + return example + +class volume_train(volume_base): + def __init__(self, data_file='data_split/splits_final2.json', phase='train', **kwargs): + super().__init__(data_file=data_file, phase=phase, **kwargs) + + +class volume_val(volume_base): + def __init__(self, data_file='data_split/splits_final2.json', phase='val', **kwargs): + super().__init__(data_file=data_file, phase=phase, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class volume_test(volume_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_base.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_base.py new file mode 100644 index 0000000000000000000000000000000000000000..fb62286c96eea7e2b0cda0d0ab58197dbae2263e --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_base.py @@ -0,0 +1,79 @@ +import os +import numpy as np +from skimage.transform import resize +from torch.utils.data import Dataset + + +class volume_base(Dataset): + def __init__(self, + data_root, + data_name, + sample=True, + sample_length=16, + data_repeat=1 + ): + + self.data_root = data_root + self.data_name = data_name + self.sample = sample + self.sample_length = sample_length + + data_root = os.path.join(data_root, data_name + '/nnUNetPlans_3d_fullres') + self.list = [os.path.join(data_name + '/nnUNetPlans_3d_fullres',f) for f in os.listdir(data_root) if f.endswith('.npz')] + self._length = len(self.list) + self._data_repeat = data_repeat + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'] + seg_npz = np.load(npz_file)['seg'] + + volume_data = data_npz[0, :, :, :] + volume_seg = seg_npz[0, :, :, :] + + volume_data = (volume_data - np.min(data_npz)) / (np.max(data_npz)-np.min(data_npz)) + if 'Abdomen' in self.data_name: + max_seg = 4 + elif 'Brain' in self.data_name: + max_seg = 3 + volume_seg = volume_seg / max_seg + + volume_data = resize(volume_data, (volume_data.shape[0],512,512)) + volume_seg = resize(volume_seg, (volume_seg.shape[0],512,512), order=0, anti_aliasing=False) + + volume_data = volume_data[:, :, :, None].astype(np.float32) + volume_seg = volume_seg[:, :, :, None].astype(np.float32) + + if self.sample: + start_id = np.random.randint(0, data_npz.shape[1]-self.sample_length) + volume_data = volume_data[start_id:start_id+self.sample_length] + volume_seg = volume_seg[start_id:start_id+self.sample_length] + + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + example['volume_data'] = volume_data * 2 - 1 + example['volume_seg'] = volume_seg * 2 - 1 + + return example + +class volume_train(volume_base): + def __init__(self, **kwargs): + super().__init__(**kwargs) + + +class volume_val(volume_base): + def __init__(self, **kwargs): + super().__init__(**kwargs) + + def __len__(self): + return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class volume_test(volume_base): + def __init__(self, **kwargs): + super().__init__(sample=False, **kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_infer.py b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_infer.py new file mode 100644 index 0000000000000000000000000000000000000000..162569c1ce5d07b9d918ac4c15a0b12949188e7b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/data/volume_dataset_infer.py @@ -0,0 +1,96 @@ +import os +import numpy as np +from skimage.transform import resize +import cv2 +from torch.utils.data import Dataset +import copy + +class volume_base(Dataset): + def __init__(self, + data_root, + data_name, + data_file, + data_repeat=1 + ): + + self.data_root = data_root + planner = 'nnUNetResEncUNetLPlans_torchres_3d_fullres' # nnUNetResEncUNetLPlans_torchres_3d_fullres nnUNetPlans_3d_fullres + data_root = os.path.join(data_root, data_name , planner) + # self.list = [os.path.join(data_name, planner ,f) for f in os.listdir(data_root) if f.endswith('.npz')] + file_list = [] + for line in open(data_file): + name = line.strip().split()[1].split('.nii.gz')[0] + name = name.split('/')[-1] + file_list.append(name) + # breakpoint() + self.list = [os.path.join(data_name, planner , i+'.npz') for i in file_list] + + self._length = len(self.list) + self.sample_length=16 + self.sample=False + self._data_repeat = data_repeat + + + def __len__(self): + return self._length * self._data_repeat + + def __getitem__(self, i): + i = i % self._length + + npz_file = os.path.join(self.data_root, self.list[i]) + data_npz = np.load(npz_file)['data'][0] # np.load(npz_file)['seg'] + seg_npz = np.load(npz_file)['seg'][0] + # breakpoint() + + if self.sample: + start_id = np.random.randint(0, data_npz.shape[0]-self.sample_length) + volume_data = data_npz[start_id:start_id+self.sample_length] + volume_seg = seg_npz[start_id:start_id+self.sample_length] + else: + volume_data=data_npz + volume_seg = seg_npz + # breakpoint() + + + volume_data[volume_data <= -175] = -175 + volume_data[volume_data >= 250] = 250 + volume_data = (volume_data+175)/425.0 + + + volume_seg = volume_seg.astype(np.float32) + foreground_mask = (volume_seg>0).astype(np.float32) + masked_data = (1-foreground_mask)*volume_data + + volume_data = volume_data[:,:,:,None].astype(np.float32) + masked_data = masked_data[:,:,:,None].astype(np.float32) + + # breakpoint() + volume_seg = resize(volume_seg, (volume_seg.shape[0],64,64), order=0, anti_aliasing=False) + # print(np.unique(resize(volume_seg, (volume_seg.shape[0],64,64), order=0, anti_aliasing=False))) + volume_seg = volume_seg[:,:,:,None].astype(np.float32) + + example = {} + example['name'] = self.list[i].split('/')[-1].split('.')[0] + # example['pos_id'] = pos_id + example['volume_data'] = volume_data * 2 - 1 + example['masked_data'] = masked_data * 2 - 1 + example['tumor_mask'] = volume_seg + # breakpoint() + return example + +class volume_train(volume_base): + def __init__(self, data_file='data_split/liver/train.txt', **kwargs): + super().__init__(data_file=data_file, **kwargs) + + +class volume_val(volume_base): + def __init__(self, data_file='data_split/liver/eval.txt', **kwargs): + super().__init__(data_file=data_file, **kwargs) + + # def __len__(self): + # return 2 if super().__len__() // 10000 < 2 else super().__len__() // 10000 + +class volume_test(volume_base): + def __init__(self, data_file='data_splits/test.txt', **kwargs): + super().__init__(**kwargs) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/lr_scheduler.py b/Abnormal-CT-Generation-MultiDisease/ldm/lr_scheduler.py new file mode 100644 index 0000000000000000000000000000000000000000..be39da9ca6dacc22bf3df9c7389bbb403a4a3ade --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/lr_scheduler.py @@ -0,0 +1,98 @@ +import numpy as np + + +class LambdaWarmUpCosineScheduler: + """ + note: use with a base_lr of 1.0 + """ + def __init__(self, warm_up_steps, lr_min, lr_max, lr_start, max_decay_steps, verbosity_interval=0): + self.lr_warm_up_steps = warm_up_steps + self.lr_start = lr_start + self.lr_min = lr_min + self.lr_max = lr_max + self.lr_max_decay_steps = max_decay_steps + self.last_lr = 0. + self.verbosity_interval = verbosity_interval + + def schedule(self, n, **kwargs): + if self.verbosity_interval > 0: + if n % self.verbosity_interval == 0: print(f"current step: {n}, recent lr-multiplier: {self.last_lr}") + if n < self.lr_warm_up_steps: + lr = (self.lr_max - self.lr_start) / self.lr_warm_up_steps * n + self.lr_start + self.last_lr = lr + return lr + else: + t = (n - self.lr_warm_up_steps) / (self.lr_max_decay_steps - self.lr_warm_up_steps) + t = min(t, 1.0) + lr = self.lr_min + 0.5 * (self.lr_max - self.lr_min) * ( + 1 + np.cos(t * np.pi)) + self.last_lr = lr + return lr + + def __call__(self, n, **kwargs): + return self.schedule(n,**kwargs) + + +class LambdaWarmUpCosineScheduler2: + """ + supports repeated iterations, configurable via lists + note: use with a base_lr of 1.0. + """ + def __init__(self, warm_up_steps, f_min, f_max, f_start, cycle_lengths, verbosity_interval=0): + assert len(warm_up_steps) == len(f_min) == len(f_max) == len(f_start) == len(cycle_lengths) + self.lr_warm_up_steps = warm_up_steps + self.f_start = f_start + self.f_min = f_min + self.f_max = f_max + self.cycle_lengths = cycle_lengths + self.cum_cycles = np.cumsum([0] + list(self.cycle_lengths)) + self.last_f = 0. + self.verbosity_interval = verbosity_interval + + def find_in_interval(self, n): + interval = 0 + for cl in self.cum_cycles[1:]: + if n <= cl: + return interval + interval += 1 + + def schedule(self, n, **kwargs): + cycle = self.find_in_interval(n) + n = n - self.cum_cycles[cycle] + if self.verbosity_interval > 0: + if n % self.verbosity_interval == 0: print(f"current step: {n}, recent lr-multiplier: {self.last_f}, " + f"current cycle {cycle}") + if n < self.lr_warm_up_steps[cycle]: + f = (self.f_max[cycle] - self.f_start[cycle]) / self.lr_warm_up_steps[cycle] * n + self.f_start[cycle] + self.last_f = f + return f + else: + t = (n - self.lr_warm_up_steps[cycle]) / (self.cycle_lengths[cycle] - self.lr_warm_up_steps[cycle]) + t = min(t, 1.0) + f = self.f_min[cycle] + 0.5 * (self.f_max[cycle] - self.f_min[cycle]) * ( + 1 + np.cos(t * np.pi)) + self.last_f = f + return f + + def __call__(self, n, **kwargs): + return self.schedule(n, **kwargs) + + +class LambdaLinearScheduler(LambdaWarmUpCosineScheduler2): + + def schedule(self, n, **kwargs): + cycle = self.find_in_interval(n) + n = n - self.cum_cycles[cycle] + if self.verbosity_interval > 0: + if n % self.verbosity_interval == 0: print(f"current step: {n}, recent lr-multiplier: {self.last_f}, " + f"current cycle {cycle}") + + if n < self.lr_warm_up_steps[cycle]: + f = (self.f_max[cycle] - self.f_start[cycle]) / self.lr_warm_up_steps[cycle] * n + self.f_start[cycle] + self.last_f = f + return f + else: + f = self.f_min[cycle] + (self.f_max[cycle] - self.f_min[cycle]) * (self.cycle_lengths[cycle] - n) / (self.cycle_lengths[cycle]) + self.last_f = f + return f + diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..cafe6e3bcf8ba4da330dc9c69f1801cfd0ed02cc Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..0298a804cc1cbc0ed3689324c2942ee241dc2083 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..d84b58ad9835b249e19129ffe50be21843d6e494 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..7aafba3d3d02193248b6b71a47a67d5ee0fab318 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/__pycache__/autoencoder.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/autoencoder.py b/Abnormal-CT-Generation-MultiDisease/ldm/models/autoencoder.py new file mode 100644 index 0000000000000000000000000000000000000000..d122549995ce2cd64092c81a58419ed4a15a02fd --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/models/autoencoder.py @@ -0,0 +1,219 @@ +import torch +import pytorch_lightning as pl +import torch.nn.functional as F +from contextlib import contextmanager + +from ldm.modules.diffusionmodules.model import Encoder, Decoder +from ldm.modules.distributions.distributions import DiagonalGaussianDistribution + +from ldm.util import instantiate_from_config +from ldm.modules.ema import LitEma + + +class AutoencoderKL(pl.LightningModule): + def __init__(self, + ddconfig, + lossconfig, + embed_dim, + ckpt_path=None, + ignore_keys=[], + image_key="image", + colorize_nlabels=None, + monitor=None, + ema_decay=None, + learn_logvar=False + ): + super().__init__() + self.learn_logvar = learn_logvar + self.image_key = image_key + self.encoder = Encoder(**ddconfig) + self.decoder = Decoder(**ddconfig) + self.loss = instantiate_from_config(lossconfig) + assert ddconfig["double_z"] + self.quant_conv = torch.nn.Conv2d(2*ddconfig["z_channels"], 2*embed_dim, 1) + self.post_quant_conv = torch.nn.Conv2d(embed_dim, ddconfig["z_channels"], 1) + self.embed_dim = embed_dim + if colorize_nlabels is not None: + assert type(colorize_nlabels)==int + self.register_buffer("colorize", torch.randn(3, colorize_nlabels, 1, 1)) + if monitor is not None: + self.monitor = monitor + + self.use_ema = ema_decay is not None + if self.use_ema: + self.ema_decay = ema_decay + assert 0. < ema_decay < 1. + self.model_ema = LitEma(self, decay=ema_decay) + print(f"Keeping EMAs of {len(list(self.model_ema.buffers()))}.") + + if ckpt_path is not None: + self.init_from_ckpt(ckpt_path, ignore_keys=ignore_keys) + + def init_from_ckpt(self, path, ignore_keys=list()): + sd = torch.load(path, map_location="cpu")["state_dict"] + keys = list(sd.keys()) + for k in keys: + for ik in ignore_keys: + if k.startswith(ik): + print("Deleting key {} from state_dict.".format(k)) + del sd[k] + self.load_state_dict(sd, strict=False) + print(f"Restored from {path}") + + @contextmanager + def ema_scope(self, context=None): + if self.use_ema: + self.model_ema.store(self.parameters()) + self.model_ema.copy_to(self) + if context is not None: + print(f"{context}: Switched to EMA weights") + try: + yield None + finally: + if self.use_ema: + self.model_ema.restore(self.parameters()) + if context is not None: + print(f"{context}: Restored training weights") + + def on_train_batch_end(self, *args, **kwargs): + if self.use_ema: + self.model_ema(self) + + def encode(self, x): + h = self.encoder(x) + moments = self.quant_conv(h) + posterior = DiagonalGaussianDistribution(moments) + return posterior + + def decode(self, z): + z = self.post_quant_conv(z) + dec = self.decoder(z) + return dec + + def forward(self, input, sample_posterior=True): + posterior = self.encode(input) + if sample_posterior: + z = posterior.sample() + else: + z = posterior.mode() + dec = self.decode(z) + return dec, posterior + + def get_input(self, batch, k): + x = batch[k] + if len(x.shape) == 3: + x = x[..., None] + x = x.permute(0, 3, 1, 2).to(memory_format=torch.contiguous_format).float() + return x + + def training_step(self, batch, batch_idx, optimizer_idx): + inputs = self.get_input(batch, self.image_key) + reconstructions, posterior = self(inputs) + + if optimizer_idx == 0: + # train encoder+decoder+logvar + aeloss, log_dict_ae = self.loss(inputs, reconstructions, posterior, optimizer_idx, self.global_step, + last_layer=self.get_last_layer(), split="train") + self.log("aeloss", aeloss, prog_bar=True, logger=True, on_step=True, on_epoch=True) + self.log_dict(log_dict_ae, prog_bar=False, logger=True, on_step=True, on_epoch=False) + return aeloss + + if optimizer_idx == 1: + # train the discriminator + discloss, log_dict_disc = self.loss(inputs, reconstructions, posterior, optimizer_idx, self.global_step, + last_layer=self.get_last_layer(), split="train") + + self.log("discloss", discloss, prog_bar=True, logger=True, on_step=True, on_epoch=True) + self.log_dict(log_dict_disc, prog_bar=False, logger=True, on_step=True, on_epoch=False) + return discloss + + def validation_step(self, batch, batch_idx): + log_dict = self._validation_step(batch, batch_idx) + with self.ema_scope(): + log_dict_ema = self._validation_step(batch, batch_idx, postfix="_ema") + return log_dict + + def _validation_step(self, batch, batch_idx, postfix=""): + inputs = self.get_input(batch, self.image_key) + reconstructions, posterior = self(inputs) + aeloss, log_dict_ae = self.loss(inputs, reconstructions, posterior, 0, self.global_step, + last_layer=self.get_last_layer(), split="val"+postfix) + + discloss, log_dict_disc = self.loss(inputs, reconstructions, posterior, 1, self.global_step, + last_layer=self.get_last_layer(), split="val"+postfix) + + self.log(f"val{postfix}/rec_loss", log_dict_ae[f"val{postfix}/rec_loss"]) + self.log_dict(log_dict_ae) + self.log_dict(log_dict_disc) + return self.log_dict + + def configure_optimizers(self): + lr = self.learning_rate + ae_params_list = list(self.encoder.parameters()) + list(self.decoder.parameters()) + list( + self.quant_conv.parameters()) + list(self.post_quant_conv.parameters()) + if self.learn_logvar: + print(f"{self.__class__.__name__}: Learning logvar") + ae_params_list.append(self.loss.logvar) + opt_ae = torch.optim.Adam(ae_params_list, + lr=lr, betas=(0.5, 0.9)) + opt_disc = torch.optim.Adam(self.loss.discriminator.parameters(), + lr=lr, betas=(0.5, 0.9)) + return [opt_ae, opt_disc], [] + + def get_last_layer(self): + return self.decoder.conv_out.weight + + @torch.no_grad() + def log_images(self, batch, only_inputs=False, log_ema=False, **kwargs): + log = dict() + x = self.get_input(batch, self.image_key) + x = x.to(self.device) + if not only_inputs: + xrec, posterior = self(x) + if x.shape[1] > 3: + # colorize with random projection + assert xrec.shape[1] > 3 + x = self.to_rgb(x) + xrec = self.to_rgb(xrec) + log["samples"] = self.decode(torch.randn_like(posterior.sample())) + log["reconstructions"] = xrec + if log_ema or self.use_ema: + with self.ema_scope(): + xrec_ema, posterior_ema = self(x) + if x.shape[1] > 3: + # colorize with random projection + assert xrec_ema.shape[1] > 3 + xrec_ema = self.to_rgb(xrec_ema) + log["samples_ema"] = self.decode(torch.randn_like(posterior_ema.sample())) + log["reconstructions_ema"] = xrec_ema + log["inputs"] = x + return log + + def to_rgb(self, x): + assert self.image_key == "segmentation" + if not hasattr(self, "colorize"): + self.register_buffer("colorize", torch.randn(3, x.shape[1], 1, 1).to(x)) + x = F.conv2d(x, weight=self.colorize) + x = 2.*(x-x.min())/(x.max()-x.min()) - 1. + return x + + +class IdentityFirstStage(torch.nn.Module): + def __init__(self, *args, vq_interface=False, **kwargs): + self.vq_interface = vq_interface + super().__init__() + + def encode(self, x, *args, **kwargs): + return x + + def decode(self, x, *args, **kwargs): + return x + + def quantize(self, x, *args, **kwargs): + if self.vq_interface: + return x, None, [None, None, None] + return x + + def forward(self, x, *args, **kwargs): + return x + diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/autoencoder__.py b/Abnormal-CT-Generation-MultiDisease/ldm/models/autoencoder__.py new file mode 100644 index 0000000000000000000000000000000000000000..6a9c4f45498561953b8085981609b2a3298a5473 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/models/autoencoder__.py @@ -0,0 +1,443 @@ +import torch +import pytorch_lightning as pl +import torch.nn.functional as F +from contextlib import contextmanager + +from taming.modules.vqvae.quantize import VectorQuantizer2 as VectorQuantizer + +from ldm.modules.diffusionmodules.model import Encoder, Decoder +from ldm.modules.distributions.distributions import DiagonalGaussianDistribution + +from ldm.util import instantiate_from_config + + +class VQModel(pl.LightningModule): + def __init__(self, + ddconfig, + lossconfig, + n_embed, + embed_dim, + ckpt_path=None, + ignore_keys=[], + image_key="image", + colorize_nlabels=None, + monitor=None, + batch_resize_range=None, + scheduler_config=None, + lr_g_factor=1.0, + remap=None, + sane_index_shape=False, # tell vector quantizer to return indices as bhw + use_ema=False + ): + super().__init__() + self.embed_dim = embed_dim + self.n_embed = n_embed + self.image_key = image_key + self.encoder = Encoder(**ddconfig) + self.decoder = Decoder(**ddconfig) + self.loss = instantiate_from_config(lossconfig) + self.quantize = VectorQuantizer(n_embed, embed_dim, beta=0.25, + remap=remap, + sane_index_shape=sane_index_shape) + self.quant_conv = torch.nn.Conv2d(ddconfig["z_channels"], embed_dim, 1) + self.post_quant_conv = torch.nn.Conv2d(embed_dim, ddconfig["z_channels"], 1) + if colorize_nlabels is not None: + assert type(colorize_nlabels)==int + self.register_buffer("colorize", torch.randn(3, colorize_nlabels, 1, 1)) + if monitor is not None: + self.monitor = monitor + self.batch_resize_range = batch_resize_range + if self.batch_resize_range is not None: + print(f"{self.__class__.__name__}: Using per-batch resizing in range {batch_resize_range}.") + + self.use_ema = use_ema + if self.use_ema: + self.model_ema = LitEma(self) + print(f"Keeping EMAs of {len(list(self.model_ema.buffers()))}.") + + if ckpt_path is not None: + self.init_from_ckpt(ckpt_path, ignore_keys=ignore_keys) + self.scheduler_config = scheduler_config + self.lr_g_factor = lr_g_factor + + @contextmanager + def ema_scope(self, context=None): + if self.use_ema: + self.model_ema.store(self.parameters()) + self.model_ema.copy_to(self) + if context is not None: + print(f"{context}: Switched to EMA weights") + try: + yield None + finally: + if self.use_ema: + self.model_ema.restore(self.parameters()) + if context is not None: + print(f"{context}: Restored training weights") + + def init_from_ckpt(self, path, ignore_keys=list()): + sd = torch.load(path, map_location="cpu")["state_dict"] + keys = list(sd.keys()) + for k in keys: + for ik in ignore_keys: + if k.startswith(ik): + print("Deleting key {} from state_dict.".format(k)) + del sd[k] + missing, unexpected = self.load_state_dict(sd, strict=False) + print(f"Restored from {path} with {len(missing)} missing and {len(unexpected)} unexpected keys") + if len(missing) > 0: + print(f"Missing Keys: {missing}") + print(f"Unexpected Keys: {unexpected}") + + def on_train_batch_end(self, *args, **kwargs): + if self.use_ema: + self.model_ema(self) + + def encode(self, x): + h = self.encoder(x) + h = self.quant_conv(h) + quant, emb_loss, info = self.quantize(h) + return quant, emb_loss, info + + def encode_to_prequant(self, x): + h = self.encoder(x) + h = self.quant_conv(h) + return h + + def decode(self, quant): + quant = self.post_quant_conv(quant) + dec = self.decoder(quant) + return dec + + def decode_code(self, code_b): + quant_b = self.quantize.embed_code(code_b) + dec = self.decode(quant_b) + return dec + + def forward(self, input, return_pred_indices=False): + quant, diff, (_,_,ind) = self.encode(input) + dec = self.decode(quant) + if return_pred_indices: + return dec, diff, ind + return dec, diff + + def get_input(self, batch, k): + x = batch[k] + if len(x.shape) == 3: + x = x[..., None] + x = x.permute(0, 3, 1, 2).to(memory_format=torch.contiguous_format).float() + if self.batch_resize_range is not None: + lower_size = self.batch_resize_range[0] + upper_size = self.batch_resize_range[1] + if self.global_step <= 4: + # do the first few batches with max size to avoid later oom + new_resize = upper_size + else: + new_resize = np.random.choice(np.arange(lower_size, upper_size+16, 16)) + if new_resize != x.shape[2]: + x = F.interpolate(x, size=new_resize, mode="bicubic") + x = x.detach() + return x + + def training_step(self, batch, batch_idx, optimizer_idx): + # https://github.com/pytorch/pytorch/issues/37142 + # try not to fool the heuristics + x = self.get_input(batch, self.image_key) + xrec, qloss, ind = self(x, return_pred_indices=True) + + if optimizer_idx == 0: + # autoencode + aeloss, log_dict_ae = self.loss(qloss, x, xrec, optimizer_idx, self.global_step, + last_layer=self.get_last_layer(), split="train", + predicted_indices=ind) + + self.log_dict(log_dict_ae, prog_bar=False, logger=True, on_step=True, on_epoch=True) + return aeloss + + if optimizer_idx == 1: + # discriminator + discloss, log_dict_disc = self.loss(qloss, x, xrec, optimizer_idx, self.global_step, + last_layer=self.get_last_layer(), split="train") + self.log_dict(log_dict_disc, prog_bar=False, logger=True, on_step=True, on_epoch=True) + return discloss + + def validation_step(self, batch, batch_idx): + log_dict = self._validation_step(batch, batch_idx) + with self.ema_scope(): + log_dict_ema = self._validation_step(batch, batch_idx, suffix="_ema") + return log_dict + + def _validation_step(self, batch, batch_idx, suffix=""): + x = self.get_input(batch, self.image_key) + xrec, qloss, ind = self(x, return_pred_indices=True) + aeloss, log_dict_ae = self.loss(qloss, x, xrec, 0, + self.global_step, + last_layer=self.get_last_layer(), + split="val"+suffix, + predicted_indices=ind + ) + + discloss, log_dict_disc = self.loss(qloss, x, xrec, 1, + self.global_step, + last_layer=self.get_last_layer(), + split="val"+suffix, + predicted_indices=ind + ) + rec_loss = log_dict_ae[f"val{suffix}/rec_loss"] + self.log(f"val{suffix}/rec_loss", rec_loss, + prog_bar=True, logger=True, on_step=False, on_epoch=True, sync_dist=True) + self.log(f"val{suffix}/aeloss", aeloss, + prog_bar=True, logger=True, on_step=False, on_epoch=True, sync_dist=True) + if version.parse(pl.__version__) >= version.parse('1.4.0'): + del log_dict_ae[f"val{suffix}/rec_loss"] + self.log_dict(log_dict_ae) + self.log_dict(log_dict_disc) + return self.log_dict + + def configure_optimizers(self): + lr_d = self.learning_rate + lr_g = self.lr_g_factor*self.learning_rate + print("lr_d", lr_d) + print("lr_g", lr_g) + opt_ae = torch.optim.Adam(list(self.encoder.parameters())+ + list(self.decoder.parameters())+ + list(self.quantize.parameters())+ + list(self.quant_conv.parameters())+ + list(self.post_quant_conv.parameters()), + lr=lr_g, betas=(0.5, 0.9)) + opt_disc = torch.optim.Adam(self.loss.discriminator.parameters(), + lr=lr_d, betas=(0.5, 0.9)) + + if self.scheduler_config is not None: + scheduler = instantiate_from_config(self.scheduler_config) + + print("Setting up LambdaLR scheduler...") + scheduler = [ + { + 'scheduler': LambdaLR(opt_ae, lr_lambda=scheduler.schedule), + 'interval': 'step', + 'frequency': 1 + }, + { + 'scheduler': LambdaLR(opt_disc, lr_lambda=scheduler.schedule), + 'interval': 'step', + 'frequency': 1 + }, + ] + return [opt_ae, opt_disc], scheduler + return [opt_ae, opt_disc], [] + + def get_last_layer(self): + return self.decoder.conv_out.weight + + def log_images(self, batch, only_inputs=False, plot_ema=False, **kwargs): + log = dict() + x = self.get_input(batch, self.image_key) + x = x.to(self.device) + if only_inputs: + log["inputs"] = x + return log + xrec, _ = self(x) + if x.shape[1] > 3: + # colorize with random projection + assert xrec.shape[1] > 3 + x = self.to_rgb(x) + xrec = self.to_rgb(xrec) + log["inputs"] = x + log["reconstructions"] = xrec + if plot_ema: + with self.ema_scope(): + xrec_ema, _ = self(x) + if x.shape[1] > 3: xrec_ema = self.to_rgb(xrec_ema) + log["reconstructions_ema"] = xrec_ema + return log + + def to_rgb(self, x): + assert self.image_key == "segmentation" + if not hasattr(self, "colorize"): + self.register_buffer("colorize", torch.randn(3, x.shape[1], 1, 1).to(x)) + x = F.conv2d(x, weight=self.colorize) + x = 2.*(x-x.min())/(x.max()-x.min()) - 1. + return x + + +class VQModelInterface(VQModel): + def __init__(self, embed_dim, *args, **kwargs): + super().__init__(embed_dim=embed_dim, *args, **kwargs) + self.embed_dim = embed_dim + + def encode(self, x): + h = self.encoder(x) + h = self.quant_conv(h) + return h + + def decode(self, h, force_not_quantize=False): + # also go through quantization layer + if not force_not_quantize: + quant, emb_loss, info = self.quantize(h) + else: + quant = h + quant = self.post_quant_conv(quant) + dec = self.decoder(quant) + return dec + + +class AutoencoderKL(pl.LightningModule): + def __init__(self, + ddconfig, + lossconfig, + embed_dim, + ckpt_path=None, + ignore_keys=[], + image_key="image", + colorize_nlabels=None, + monitor=None, + ): + super().__init__() + self.image_key = image_key + self.encoder = Encoder(**ddconfig) + self.decoder = Decoder(**ddconfig) + self.loss = instantiate_from_config(lossconfig) + assert ddconfig["double_z"] + self.quant_conv = torch.nn.Conv2d(2*ddconfig["z_channels"], 2*embed_dim, 1) + self.post_quant_conv = torch.nn.Conv2d(embed_dim, ddconfig["z_channels"], 1) + self.embed_dim = embed_dim + if colorize_nlabels is not None: + assert type(colorize_nlabels)==int + self.register_buffer("colorize", torch.randn(3, colorize_nlabels, 1, 1)) + if monitor is not None: + self.monitor = monitor + if ckpt_path is not None: + self.init_from_ckpt(ckpt_path, ignore_keys=ignore_keys) + + def init_from_ckpt(self, path, ignore_keys=list()): + sd = torch.load(path, map_location="cpu")["state_dict"] + keys = list(sd.keys()) + for k in keys: + for ik in ignore_keys: + if k.startswith(ik): + print("Deleting key {} from state_dict.".format(k)) + del sd[k] + self.load_state_dict(sd, strict=False) + print(f"Restored from {path}") + + def encode(self, x): + h = self.encoder(x) + moments = self.quant_conv(h) + posterior = DiagonalGaussianDistribution(moments) + return posterior + + def decode(self, z): + z = self.post_quant_conv(z) + dec = self.decoder(z) + return dec + + def forward(self, input, sample_posterior=True): + posterior = self.encode(input) + if sample_posterior: + z = posterior.sample() + else: + z = posterior.mode() + dec = self.decode(z) + return dec, posterior + + def get_input(self, batch, k): + x = batch[k] + if len(x.shape) == 3: + x = x[..., None] + x = x.permute(0, 3, 1, 2).to(memory_format=torch.contiguous_format).float() + return x + + def training_step(self, batch, batch_idx, optimizer_idx): + inputs = self.get_input(batch, self.image_key) + reconstructions, posterior = self(inputs) + + if optimizer_idx == 0: + # train encoder+decoder+logvar + aeloss, log_dict_ae = self.loss(inputs, reconstructions, posterior, optimizer_idx, self.global_step, + last_layer=self.get_last_layer(), split="train") + self.log("aeloss", aeloss, prog_bar=True, logger=True, on_step=True, on_epoch=True) + self.log_dict(log_dict_ae, prog_bar=False, logger=True, on_step=True, on_epoch=False) + return aeloss + + if optimizer_idx == 1: + # train the discriminator + discloss, log_dict_disc = self.loss(inputs, reconstructions, posterior, optimizer_idx, self.global_step, + last_layer=self.get_last_layer(), split="train") + + self.log("discloss", discloss, prog_bar=True, logger=True, on_step=True, on_epoch=True) + self.log_dict(log_dict_disc, prog_bar=False, logger=True, on_step=True, on_epoch=False) + return discloss + + def validation_step(self, batch, batch_idx): + inputs = self.get_input(batch, self.image_key) + reconstructions, posterior = self(inputs) + aeloss, log_dict_ae = self.loss(inputs, reconstructions, posterior, 0, self.global_step, + last_layer=self.get_last_layer(), split="val") + + discloss, log_dict_disc = self.loss(inputs, reconstructions, posterior, 1, self.global_step, + last_layer=self.get_last_layer(), split="val") + + self.log("val/rec_loss", log_dict_ae["val/rec_loss"]) + self.log_dict(log_dict_ae) + self.log_dict(log_dict_disc) + return self.log_dict + + def configure_optimizers(self): + lr = self.learning_rate + opt_ae = torch.optim.Adam(list(self.encoder.parameters())+ + list(self.decoder.parameters())+ + list(self.quant_conv.parameters())+ + list(self.post_quant_conv.parameters()), + lr=lr, betas=(0.5, 0.9)) + opt_disc = torch.optim.Adam(self.loss.discriminator.parameters(), + lr=lr, betas=(0.5, 0.9)) + return [opt_ae, opt_disc], [] + + def get_last_layer(self): + return self.decoder.conv_out.weight + + @torch.no_grad() + def log_images(self, batch, only_inputs=False, **kwargs): + log = dict() + x = self.get_input(batch, self.image_key) + x = x.to(self.device) + if not only_inputs: + xrec, posterior = self(x) + if x.shape[1] > 3: + # colorize with random projection + assert xrec.shape[1] > 3 + x = self.to_rgb(x) + xrec = self.to_rgb(xrec) + log["samples"] = self.decode(torch.randn_like(posterior.sample())) + log["reconstructions"] = xrec + log["inputs"] = x + return log + + def to_rgb(self, x): + assert self.image_key == "segmentation" + if not hasattr(self, "colorize"): + self.register_buffer("colorize", torch.randn(3, x.shape[1], 1, 1).to(x)) + x = F.conv2d(x, weight=self.colorize) + x = 2.*(x-x.min())/(x.max()-x.min()) - 1. + return x + + +class IdentityFirstStage(torch.nn.Module): + def __init__(self, *args, vq_interface=False, **kwargs): + self.vq_interface = vq_interface # TODO: Should be true by default but check to not break older stuff + super().__init__() + + def encode(self, x, *args, **kwargs): + return x + + def decode(self, x, *args, **kwargs): + return x + + def quantize(self, x, *args, **kwargs): + if self.vq_interface: + return x, None, [None, None, None] + return x + + def forward(self, x, *args, **kwargs): + return x diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__init__.py b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..20c2790c41a78f62d957c67c912d153041bb3f0f Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..e35472c178a6a569a4989cce5ec27368017a1583 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..fdae1ed7cee83d48e8c8da0502f9d6d048099152 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..14543402bb3594bf296fbfdcc75a35fbe7524e04 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/__init__.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..acf2cda35511b181e6be3fa5dcc591cc4ff1c68f Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..8c8b5ebfbbcfedd58e62ef0e184936f1f6b12642 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..0316ec9509ea087b0d264008c784fe1f519842fc Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..6df062eefbfda3ea08fe823a955c47a33e0f3b18 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddim.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..29d3c46988984522cb66d307d64d4149685f89bb Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..a8b71a36a5fb26dabdf439e74a3b6c3421424d54 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..d2c56b2eaba52467184ba49ad58fe325add0e0ad Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..8bdf91d2b951a4f64cff363219e56c64011385f9 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..b1fb02b7a174656b7549c4f04e7d42176f714a98 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..c0cfebea6c912269ad5524fec1bd9ce1f5d366a4 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..d76520095009ae07d90fea26562054985164cd81 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_pseudo3D.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_recon.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_recon.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..5e3c632485f61af3ee9dacdf071f27d03cbbfdcc Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_recon.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_recon_vq.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_recon_vq.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..2da866830c272f6486c94762e9c5937e0d1d67a7 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/__pycache__/ddpm_recon_vq.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddim.py b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddim.py new file mode 100644 index 0000000000000000000000000000000000000000..9f8dc584daffe533c0a02a721daa35918a9487f9 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddim.py @@ -0,0 +1,365 @@ +"""SAMPLING ONLY.""" + +import torch +import numpy as np +from tqdm import tqdm +from einops import rearrange + +from ldm.modules.diffusionmodules.util import make_ddim_sampling_parameters, make_ddim_timesteps, noise_like, extract_into_tensor + + +class DDIMSampler(object): + def __init__(self, model, schedule="linear", **kwargs): + super().__init__() + self.model = model + self.ddpm_num_timesteps = model.num_timesteps + self.schedule = schedule + + def register_buffer(self, name, attr): + if type(attr) == torch.Tensor: + if attr.device != torch.device("cuda"): + attr = attr.to(torch.device("cuda")) + setattr(self, name, attr) + + def make_schedule(self, ddim_num_steps, ddim_discretize="uniform", ddim_eta=0., verbose=True): + self.ddim_timesteps = make_ddim_timesteps(ddim_discr_method=ddim_discretize, num_ddim_timesteps=ddim_num_steps, + num_ddpm_timesteps=self.ddpm_num_timesteps,verbose=verbose) + alphas_cumprod = self.model.alphas_cumprod + assert alphas_cumprod.shape[0] == self.ddpm_num_timesteps, 'alphas have to be defined for each timestep' + to_torch = lambda x: x.clone().detach().to(torch.float32).to(self.model.device) + + self.register_buffer('betas', to_torch(self.model.betas)) + self.register_buffer('alphas_cumprod', to_torch(alphas_cumprod)) + self.register_buffer('alphas_cumprod_prev', to_torch(self.model.alphas_cumprod_prev)) + + # calculations for diffusion q(x_t | x_{t-1}) and others + self.register_buffer('sqrt_alphas_cumprod', to_torch(np.sqrt(alphas_cumprod.cpu()))) + self.register_buffer('sqrt_one_minus_alphas_cumprod', to_torch(np.sqrt(1. - alphas_cumprod.cpu()))) + self.register_buffer('log_one_minus_alphas_cumprod', to_torch(np.log(1. - alphas_cumprod.cpu()))) + self.register_buffer('sqrt_recip_alphas_cumprod', to_torch(np.sqrt(1. / alphas_cumprod.cpu()))) + self.register_buffer('sqrt_recipm1_alphas_cumprod', to_torch(np.sqrt(1. / alphas_cumprod.cpu() - 1))) + + # ddim sampling parameters + ddim_sigmas, ddim_alphas, ddim_alphas_prev = make_ddim_sampling_parameters(alphacums=alphas_cumprod.cpu(), + ddim_timesteps=self.ddim_timesteps, + eta=ddim_eta,verbose=verbose) + self.register_buffer('ddim_sigmas', ddim_sigmas) + self.register_buffer('ddim_alphas', ddim_alphas) + self.register_buffer('ddim_alphas_prev', ddim_alphas_prev) + self.register_buffer('ddim_sqrt_one_minus_alphas', np.sqrt(1. - ddim_alphas)) + sigmas_for_original_sampling_steps = ddim_eta * torch.sqrt( + (1 - self.alphas_cumprod_prev) / (1 - self.alphas_cumprod) * ( + 1 - self.alphas_cumprod / self.alphas_cumprod_prev)) + self.register_buffer('ddim_sigmas_for_original_num_steps', sigmas_for_original_sampling_steps) + + @torch.no_grad() + def sample(self, + S, + batch_size, + shape, + conditioning=None, + callback=None, + normals_sequence=None, + img_callback=None, + quantize_x0=False, + eta=0., + mask=None, + x0=None, + temperature=1., + noise_dropout=0., + score_corrector=None, + corrector_kwargs=None, + verbose=True, + x_T=None, + log_every_t=100, + unconditional_guidance_scale=1., + unconditional_conditioning=None, # this has to come in the same format as the conditioning, # e.g. as encoded tokens, ... + dynamic_threshold=None, + ucg_schedule=None, + previous=None, + previous_reverse=False, + **kwargs + ): + if conditioning is not None: + if isinstance(conditioning, dict): + ctmp = conditioning[list(conditioning.keys())[0]] + while isinstance(ctmp, list): ctmp = ctmp[0] + cbs = ctmp.shape[0] + if cbs != batch_size: + print(f"Warning: Got {cbs} conditionings but batch-size is {batch_size}") + + elif isinstance(conditioning, list): + for ctmp in conditioning: + if ctmp.shape[0] != batch_size: + print(f"Warning: Got {cbs} conditionings but batch-size is {batch_size}") + + else: + if conditioning.shape[0] != batch_size: + print(f"Warning: Got {conditioning.shape[0]} conditionings but batch-size is {batch_size}") + + self.make_schedule(ddim_num_steps=S, ddim_eta=eta, verbose=verbose) + # sampling + C, H, W = shape + size = (batch_size, C, H, W) + print(f'Data shape for DDIM sampling is {size}, eta {eta}') + # breakpoint() + samples, intermediates = self.ddim_sampling(conditioning, size, + callback=callback, + img_callback=img_callback, + quantize_denoised=quantize_x0, + mask=mask, x0=x0, + ddim_use_original_steps=False, + noise_dropout=noise_dropout, + temperature=temperature, + score_corrector=score_corrector, + corrector_kwargs=corrector_kwargs, + x_T=x_T, + log_every_t=log_every_t, + previous=previous, + previous_reverse=previous_reverse, + unconditional_guidance_scale=unconditional_guidance_scale, + unconditional_conditioning=unconditional_conditioning, + dynamic_threshold=dynamic_threshold, + ucg_schedule=ucg_schedule + ) + return samples, intermediates + + @torch.no_grad() + def ddim_sampling(self, cond, shape, + x_T=None, ddim_use_original_steps=False, + callback=None, timesteps=None, quantize_denoised=False, + mask=None, x0=None, img_callback=None, log_every_t=100, + temperature=1., noise_dropout=0., score_corrector=None, corrector_kwargs=None, + unconditional_guidance_scale=1., unconditional_conditioning=None, dynamic_threshold=None, + ucg_schedule=None, previous=None, previous_reverse=False): + device = self.model.betas.device + b = shape[0] + if x_T is None: + img = torch.randn(shape, device=device) + else: + img = x_T + + if previous is not None: + img = rearrange(img, '(b t) c h w -> b t c h w', b=previous.shape[0]) + alphas = self.model.alphas_cumprod + c0 = torch.sqrt(alphas[-1]) + c1 = torch.sqrt(1-alphas[-1]) + previous_noisy = c0*previous+c1*torch.randn_like(previous) + if not previous_reverse: + img[:, :previous.shape[1]] = previous_noisy + else: + img[:, -previous.shape[1]:] = previous_noisy + img = rearrange(img, 'b t c h w -> (b t) c h w') + + if timesteps is None: + timesteps = self.ddpm_num_timesteps if ddim_use_original_steps else self.ddim_timesteps + elif timesteps is not None and not ddim_use_original_steps: + subset_end = int(min(timesteps / self.ddim_timesteps.shape[0], 1) * self.ddim_timesteps.shape[0]) - 1 + timesteps = self.ddim_timesteps[:subset_end] + + intermediates = {'x_inter': [img], 'pred_x0': [img]} + time_range = reversed(range(0,timesteps)) if ddim_use_original_steps else np.flip(timesteps) + total_steps = timesteps if ddim_use_original_steps else timesteps.shape[0] + print(f"Running DDIM Sampling with {total_steps} timesteps") + + iterator = tqdm(time_range, desc='DDIM Sampler', total=total_steps) + + for i, step in enumerate(iterator): + index = total_steps - i - 1 + ts = torch.full((b,), step, device=device, dtype=torch.long) + + if mask is not None: + assert x0 is not None + img_orig = self.model.q_sample(x0, ts) + img = img_orig * mask + (1. - mask) * img + + if ucg_schedule is not None: + assert len(ucg_schedule) == len(time_range) + unconditional_guidance_scale = ucg_schedule[i] + + outs = self.p_sample_ddim(img, cond, ts, index=index, use_original_steps=ddim_use_original_steps, + quantize_denoised=quantize_denoised, temperature=temperature, + noise_dropout=noise_dropout, score_corrector=score_corrector, + corrector_kwargs=corrector_kwargs, + unconditional_guidance_scale=unconditional_guidance_scale, + unconditional_conditioning=unconditional_conditioning, + dynamic_threshold=dynamic_threshold) + img, pred_x0 = outs + if callback: callback(i) + if img_callback: img_callback(pred_x0, i) + + if previous is not None: + img = rearrange(img, '(b t) c h w -> b t c h w', b=previous.shape[0]) + alphas = self.model.alphas_cumprod + c0 = torch.sqrt(alphas[index]) + c1 = torch.sqrt(1-alphas[index]) + previous_noisy = c0*previous+c1*torch.randn_like(previous) + if not previous_reverse: + img[:, :previous.shape[1]] = previous_noisy + else: + img[:, -previous.shape[1]:] = previous_noisy + img = rearrange(img, 'b t c h w -> (b t) c h w') + + if index % log_every_t == 0 or index == total_steps - 1: + intermediates['x_inter'].append(img) + intermediates['pred_x0'].append(pred_x0) + + return img, intermediates + + @torch.no_grad() + def p_sample_ddim(self, x, c, t, index, repeat_noise=False, use_original_steps=False, quantize_denoised=False, + temperature=1., noise_dropout=0., score_corrector=None, corrector_kwargs=None, + unconditional_guidance_scale=1., unconditional_conditioning=None, + dynamic_threshold=None): + b, *_, device = *x.shape, x.device + + if unconditional_conditioning is None or unconditional_guidance_scale == 1.: + model_output = self.model.apply_model(x, t, c) + else: + x_in = torch.cat([x] * 2) + t_in = torch.cat([t] * 2) + if isinstance(c, dict): + assert isinstance(unconditional_conditioning, dict) + c_in = dict() + for k in c: + if isinstance(c[k], list): + c_in[k] = [torch.cat([ + unconditional_conditioning[k][i], + c[k][i]]) for i in range(len(c[k]))] + else: + c_in[k] = torch.cat([ + unconditional_conditioning[k], + c[k]]) + elif isinstance(c, list): + c_in = list() + assert isinstance(unconditional_conditioning, list) + for i in range(len(c)): + c_in.append(torch.cat([unconditional_conditioning[i], c[i]])) + else: + c_in = torch.cat([unconditional_conditioning, c]) + model_uncond, model_t = self.model.apply_model(x_in, t_in, c_in).chunk(2) + model_output = model_uncond + unconditional_guidance_scale * (model_t - model_uncond) + + if self.model.parameterization == "v": + e_t = self.model.predict_eps_from_z_and_v(x, t, model_output) + else: + e_t = model_output + + if score_corrector is not None: + assert self.model.parameterization == "eps", 'not implemented' + e_t = score_corrector.modify_score(self.model, e_t, x, t, c, **corrector_kwargs) + + alphas = self.model.alphas_cumprod if use_original_steps else self.ddim_alphas + alphas_prev = self.model.alphas_cumprod_prev if use_original_steps else self.ddim_alphas_prev + sqrt_one_minus_alphas = self.model.sqrt_one_minus_alphas_cumprod if use_original_steps else self.ddim_sqrt_one_minus_alphas + sigmas = self.model.ddim_sigmas_for_original_num_steps if use_original_steps else self.ddim_sigmas + # select parameters corresponding to the currently considered timestep + a_t = torch.full((b, 1, 1, 1), alphas[index], device=device) + a_prev = torch.full((b, 1, 1, 1), alphas_prev[index], device=device) + sigma_t = torch.full((b, 1, 1, 1), sigmas[index], device=device) + sqrt_one_minus_at = torch.full((b, 1, 1, 1), sqrt_one_minus_alphas[index],device=device) + + # current prediction for x_0 + if self.model.parameterization != "v": + pred_x0 = (x - sqrt_one_minus_at * e_t) / a_t.sqrt() + else: + pred_x0 = self.model.predict_start_from_z_and_v(x, t, model_output) + + if quantize_denoised: + pred_x0, _, *_ = self.model.first_stage_model.quantize(pred_x0) + + if dynamic_threshold is not None: + raise NotImplementedError() + + # direction pointing to x_t + dir_xt = (1. - a_prev - sigma_t**2).sqrt() * e_t + noise = sigma_t * noise_like(x.shape, device, repeat_noise) * temperature + if noise_dropout > 0.: + noise = torch.nn.functional.dropout(noise, p=noise_dropout) + x_prev = a_prev.sqrt() * pred_x0 + dir_xt + noise + return x_prev, pred_x0 + + @torch.no_grad() + def encode(self, x0, c, t_enc, use_original_steps=False, return_intermediates=None, + unconditional_guidance_scale=1.0, unconditional_conditioning=None, callback=None): + num_reference_steps = self.ddpm_num_timesteps if use_original_steps else self.ddim_timesteps.shape[0] + + assert t_enc <= num_reference_steps + num_steps = t_enc + + if use_original_steps: + alphas_next = self.alphas_cumprod[:num_steps] + alphas = self.alphas_cumprod_prev[:num_steps] + else: + alphas_next = self.ddim_alphas[:num_steps] + alphas = torch.tensor(self.ddim_alphas_prev[:num_steps]) + + x_next = x0 + intermediates = [] + inter_steps = [] + for i in tqdm(range(num_steps), desc='Encoding Image'): + t = torch.full((x0.shape[0],), i, device=self.model.device, dtype=torch.long) + if unconditional_guidance_scale == 1.: + noise_pred = self.model.apply_model(x_next, t, c) + else: + assert unconditional_conditioning is not None + e_t_uncond, noise_pred = torch.chunk( + self.model.apply_model(torch.cat((x_next, x_next)), torch.cat((t, t)), + torch.cat((unconditional_conditioning, c))), 2) + noise_pred = e_t_uncond + unconditional_guidance_scale * (noise_pred - e_t_uncond) + + xt_weighted = (alphas_next[i] / alphas[i]).sqrt() * x_next + weighted_noise_pred = alphas_next[i].sqrt() * ( + (1 / alphas_next[i] - 1).sqrt() - (1 / alphas[i] - 1).sqrt()) * noise_pred + x_next = xt_weighted + weighted_noise_pred + if return_intermediates and i % ( + num_steps // return_intermediates) == 0 and i < num_steps - 1: + intermediates.append(x_next) + inter_steps.append(i) + elif return_intermediates and i >= num_steps - 2: + intermediates.append(x_next) + inter_steps.append(i) + if callback: callback(i) + + out = {'x_encoded': x_next, 'intermediate_steps': inter_steps} + if return_intermediates: + out.update({'intermediates': intermediates}) + return x_next, out + + @torch.no_grad() + def stochastic_encode(self, x0, t, use_original_steps=False, noise=None): + # fast, but does not allow for exact reconstruction + # t serves as an index to gather the correct alphas + if use_original_steps: + sqrt_alphas_cumprod = self.sqrt_alphas_cumprod + sqrt_one_minus_alphas_cumprod = self.sqrt_one_minus_alphas_cumprod + else: + sqrt_alphas_cumprod = torch.sqrt(self.ddim_alphas) + sqrt_one_minus_alphas_cumprod = self.ddim_sqrt_one_minus_alphas + + if noise is None: + noise = torch.randn_like(x0) + return (extract_into_tensor(sqrt_alphas_cumprod, t, x0.shape) * x0 + + extract_into_tensor(sqrt_one_minus_alphas_cumprod, t, x0.shape) * noise) + + @torch.no_grad() + def decode(self, x_latent, cond, t_start, unconditional_guidance_scale=1.0, unconditional_conditioning=None, + use_original_steps=False, callback=None): + + timesteps = np.arange(self.ddpm_num_timesteps) if use_original_steps else self.ddim_timesteps + timesteps = timesteps[:t_start] + + time_range = np.flip(timesteps) + total_steps = timesteps.shape[0] + print(f"Running DDIM Sampling with {total_steps} timesteps") + + iterator = tqdm(time_range, desc='Decoding image', total=total_steps) + x_dec = x_latent + for i, step in enumerate(iterator): + index = total_steps - i - 1 + ts = torch.full((x_latent.shape[0],), step, device=x_latent.device, dtype=torch.long) + x_dec, _ = self.p_sample_ddim(x_dec, cond, ts, index=index, use_original_steps=use_original_steps, + unconditional_guidance_scale=unconditional_guidance_scale, + unconditional_conditioning=unconditional_conditioning) + if callback: callback(i) + return x_dec \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm.py b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm.py new file mode 100644 index 0000000000000000000000000000000000000000..a48c20da2b526c989996ce755d280f3c9c63d2ef --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm.py @@ -0,0 +1,1447 @@ +""" +wild mixture of +https://github.com/lucidrains/denoising-diffusion-pytorch/blob/7706bdfc6f527f58d33f84b7b522e61e6e3164b3/denoising_diffusion_pytorch/denoising_diffusion_pytorch.py +https://github.com/openai/improved-diffusion/blob/e94489283bb876ac1477d5dd7709bbbd2d9902ce/improved_diffusion/gaussian_diffusion.py +https://github.com/CompVis/taming-transformers +-- merci +""" + +import torch +import torch.nn as nn +import numpy as np + +import enum + +# 保存原始实现(可选,用于调试或回滚) +_enum_format_orig = enum.Enum.__format__ + +def _enum_format_value(self, format_spec: str) -> str: + # 如果指定了 format 规范,就交给 value 本身的格式化;否则直接返回 value + return format(self.value, format_spec) if format_spec else str(self.value) + +# 全局替换 Enum.__format__ +enum.Enum.__format__ = _enum_format_value + +import pytorch_lightning as pl +from torch.optim.lr_scheduler import LambdaLR +from einops import rearrange, repeat +from contextlib import contextmanager, nullcontext +from functools import partial +import itertools +from tqdm import tqdm +from torchvision.utils import make_grid +from pytorch_lightning.utilities.distributed import rank_zero_only +from omegaconf import ListConfig + +from ldm.util import log_txt_as_img, exists, default, ismap, isimage, mean_flat, count_params, instantiate_from_config +from ldm.modules.ema import LitEma +from ldm.modules.distributions.distributions import normal_kl, DiagonalGaussianDistribution +from ldm.models.autoencoder import IdentityFirstStage, AutoencoderKL +from ldm.modules.diffusionmodules.util import make_beta_schedule, extract_into_tensor, noise_like +from ldm.models.diffusion.ddim import DDIMSampler +from transformers import BertTokenizer,BertModel + +__conditioning_keys__ = {'concat': 'c_concat', + 'crossattn': 'c_crossattn', + 'adm': 'y'} + + +def disabled_train(self, mode=True): + """Overwrite model.train with this function to make sure train/eval mode + does not change anymore.""" + return self + + +def uniform_on_device(r1, r2, shape, device): + return (r1 - r2) * torch.rand(*shape, device=device) + r2 + + +class DDPM(pl.LightningModule): + # classic DDPM with Gaussian diffusion, in image space + def __init__(self, + unet_config, + timesteps=1000, + beta_schedule="linear", + loss_type="l2", + ckpt_path=None, + ignore_keys=[], + load_only_unet=False, + monitor="val/loss", + use_ema=True, + first_stage_key="image", + image_size=256, + channels=3, + log_every_t=100, + clip_denoised=True, + linear_start=1e-4, + linear_end=2e-2, + cosine_s=8e-3, + given_betas=None, + original_elbo_weight=0., + v_posterior=0., # weight for choosing posterior variance as sigma = (1-v) * beta_tilde + v * beta + l_simple_weight=1., + conditioning_key=None, + parameterization="eps", # all assuming fixed variance schedules + scheduler_config=None, + use_positional_encodings=False, + learn_logvar=False, + logvar_init=0., + make_it_fit=False, + ucg_training=None, + reset_ema=False, + reset_num_ema_updates=False, + ): + super().__init__() + assert parameterization in ["eps", "x0", "v"], 'currently only supporting "eps" and "x0" and "v"' + self.parameterization = parameterization + print(f"{self.__class__.__name__}: Running in {self.parameterization}-prediction mode") + self.cond_stage_model = None + self.clip_denoised = clip_denoised + self.log_every_t = log_every_t + self.first_stage_key = first_stage_key + self.image_size = image_size # try conv? + self.channels = channels + + self.use_positional_encodings = use_positional_encodings + self.model = DiffusionWrapper(unet_config, conditioning_key) + count_params(self.model, verbose=True) + self.use_ema = use_ema + if self.use_ema: + self.model_ema = LitEma(self.model) + print(f"Keeping EMAs of {len(list(self.model_ema.buffers()))}.") + + self.use_scheduler = scheduler_config is not None + if self.use_scheduler: + self.scheduler_config = scheduler_config + + self.v_posterior = v_posterior + self.original_elbo_weight = original_elbo_weight + self.l_simple_weight = l_simple_weight + + if monitor is not None: + self.monitor = monitor + self.make_it_fit = make_it_fit + if reset_ema: assert exists(ckpt_path) + if ckpt_path is not None: + self.init_from_ckpt(ckpt_path, ignore_keys=ignore_keys, only_model=load_only_unet) + if reset_ema: + assert self.use_ema + print(f"Resetting ema to pure model weights. This is useful when restoring from an ema-only checkpoint.") + self.model_ema = LitEma(self.model) + if reset_num_ema_updates: + print(" +++++++++++ WARNING: RESETTING NUM_EMA UPDATES TO ZERO +++++++++++ ") + assert self.use_ema + self.model_ema.reset_num_updates() + + self.register_schedule(given_betas=given_betas, beta_schedule=beta_schedule, timesteps=timesteps, + linear_start=linear_start, linear_end=linear_end, cosine_s=cosine_s) + + self.loss_type = loss_type + + self.learn_logvar = learn_logvar + logvar = torch.full(fill_value=logvar_init, size=(self.num_timesteps,)) + if self.learn_logvar: + self.logvar = nn.Parameter(self.logvar, requires_grad=True) + else: + self.register_buffer('logvar', logvar) + + self.ucg_training = ucg_training or dict() + if self.ucg_training: + self.ucg_prng = np.random.RandomState() + + def register_schedule(self, given_betas=None, beta_schedule="linear", timesteps=1000, + linear_start=1e-4, linear_end=2e-2, cosine_s=8e-3): + if exists(given_betas): + betas = given_betas + else: + betas = make_beta_schedule(beta_schedule, timesteps, linear_start=linear_start, linear_end=linear_end, + cosine_s=cosine_s) + alphas = 1. - betas + alphas_cumprod = np.cumprod(alphas, axis=0) + alphas_cumprod_prev = np.append(1., alphas_cumprod[:-1]) + + timesteps, = betas.shape + self.num_timesteps = int(timesteps) + self.linear_start = linear_start + self.linear_end = linear_end + assert alphas_cumprod.shape[0] == self.num_timesteps, 'alphas have to be defined for each timestep' + + to_torch = partial(torch.tensor, dtype=torch.float32) + + self.register_buffer('betas', to_torch(betas)) + self.register_buffer('alphas_cumprod', to_torch(alphas_cumprod)) + self.register_buffer('alphas_cumprod_prev', to_torch(alphas_cumprod_prev)) + + # calculations for diffusion q(x_t | x_{t-1}) and others + self.register_buffer('sqrt_alphas_cumprod', to_torch(np.sqrt(alphas_cumprod))) + self.register_buffer('sqrt_one_minus_alphas_cumprod', to_torch(np.sqrt(1. - alphas_cumprod))) + self.register_buffer('log_one_minus_alphas_cumprod', to_torch(np.log(1. - alphas_cumprod))) + self.register_buffer('sqrt_recip_alphas_cumprod', to_torch(np.sqrt(1. / alphas_cumprod))) + self.register_buffer('sqrt_recipm1_alphas_cumprod', to_torch(np.sqrt(1. / alphas_cumprod - 1))) + + # calculations for posterior q(x_{t-1} | x_t, x_0) + posterior_variance = (1 - self.v_posterior) * betas * (1. - alphas_cumprod_prev) / ( + 1. - alphas_cumprod) + self.v_posterior * betas + # above: equal to 1. / (1. / (1. - alpha_cumprod_tm1) + alpha_t / beta_t) + self.register_buffer('posterior_variance', to_torch(posterior_variance)) + # below: log calculation clipped because the posterior variance is 0 at the beginning of the diffusion chain + self.register_buffer('posterior_log_variance_clipped', to_torch(np.log(np.maximum(posterior_variance, 1e-20)))) + self.register_buffer('posterior_mean_coef1', to_torch( + betas * np.sqrt(alphas_cumprod_prev) / (1. - alphas_cumprod))) + self.register_buffer('posterior_mean_coef2', to_torch( + (1. - alphas_cumprod_prev) * np.sqrt(alphas) / (1. - alphas_cumprod))) + + if self.parameterization == "eps": + lvlb_weights = self.betas ** 2 / ( + 2 * self.posterior_variance * to_torch(alphas) * (1 - self.alphas_cumprod)) + elif self.parameterization == "x0": + lvlb_weights = 0.5 * np.sqrt(torch.Tensor(alphas_cumprod)) / (2. * 1 - torch.Tensor(alphas_cumprod)) + elif self.parameterization == "v": + lvlb_weights = torch.ones_like(self.betas ** 2 / ( + 2 * self.posterior_variance * to_torch(alphas) * (1 - self.alphas_cumprod))) + else: + raise NotImplementedError("mu not supported") + lvlb_weights[0] = lvlb_weights[1] + self.register_buffer('lvlb_weights', lvlb_weights, persistent=False) + assert not torch.isnan(self.lvlb_weights).all() + + @contextmanager + def ema_scope(self, context=None): + if self.use_ema: + self.model_ema.store(self.model.parameters()) + self.model_ema.copy_to(self.model) + if context is not None: + print(f"{context}: Switched to EMA weights") + try: + yield None + finally: + if self.use_ema: + self.model_ema.restore(self.model.parameters()) + if context is not None: + print(f"{context}: Restored training weights") + + @torch.no_grad() + def init_from_ckpt(self, path, ignore_keys=list(), only_model=False): + sd = torch.load(path, map_location="cpu") + if "state_dict" in list(sd.keys()): + sd = sd["state_dict"] + keys = list(sd.keys()) + for k in keys: + for ik in ignore_keys: + if k.startswith(ik): + print("Deleting key {} from state_dict.".format(k)) + del sd[k] + if self.make_it_fit: + n_params = len([name for name, _ in + itertools.chain(self.named_parameters(), + self.named_buffers())]) + for name, param in tqdm( + itertools.chain(self.named_parameters(), + self.named_buffers()), + desc="Fitting old weights to new weights", + total=n_params + ): + if not name in sd: + continue + old_shape = sd[name].shape + new_shape = param.shape + assert len(old_shape) == len(new_shape) + if len(new_shape) > 2: + # we only modify first two axes + assert new_shape[2:] == old_shape[2:] + # assumes first axis corresponds to output dim + if not new_shape == old_shape: + new_param = param.clone() + old_param = sd[name] + if len(new_shape) == 1: + for i in range(new_param.shape[0]): + new_param[i] = old_param[i % old_shape[0]] + elif len(new_shape) >= 2: + for i in range(new_param.shape[0]): + for j in range(new_param.shape[1]): + new_param[i, j] = old_param[i % old_shape[0], j % old_shape[1]] + + n_used_old = torch.ones(old_shape[1]) + for j in range(new_param.shape[1]): + n_used_old[j % old_shape[1]] += 1 + n_used_new = torch.zeros(new_shape[1]) + for j in range(new_param.shape[1]): + n_used_new[j] = n_used_old[j % old_shape[1]] + + n_used_new = n_used_new[None, :] + while len(n_used_new.shape) < len(new_shape): + n_used_new = n_used_new.unsqueeze(-1) + new_param /= n_used_new + + sd[name] = new_param + + missing, unexpected = self.load_state_dict(sd, strict=False) if not only_model else self.model.load_state_dict( + sd, strict=False) + print(f"Restored from {path} with {len(missing)} missing and {len(unexpected)} unexpected keys") + if len(missing) > 0: + print(f"Missing Keys:\n {missing}") + if len(unexpected) > 0: + print(f"\nUnexpected Keys:\n {unexpected}") + + def q_mean_variance(self, x_start, t): + """ + Get the distribution q(x_t | x_0). + :param x_start: the [N x C x ...] tensor of noiseless inputs. + :param t: the number of diffusion steps (minus 1). Here, 0 means one step. + :return: A tuple (mean, variance, log_variance), all of x_start's shape. + """ + mean = (extract_into_tensor(self.sqrt_alphas_cumprod, t, x_start.shape) * x_start) + variance = extract_into_tensor(1.0 - self.alphas_cumprod, t, x_start.shape) + log_variance = extract_into_tensor(self.log_one_minus_alphas_cumprod, t, x_start.shape) + return mean, variance, log_variance + + def predict_start_from_noise(self, x_t, t, noise): + return ( + extract_into_tensor(self.sqrt_recip_alphas_cumprod, t, x_t.shape) * x_t - + extract_into_tensor(self.sqrt_recipm1_alphas_cumprod, t, x_t.shape) * noise + ) + + def predict_start_from_z_and_v(self, x_t, t, v): + # self.register_buffer('sqrt_alphas_cumprod', to_torch(np.sqrt(alphas_cumprod))) + # self.register_buffer('sqrt_one_minus_alphas_cumprod', to_torch(np.sqrt(1. - alphas_cumprod))) + return ( + extract_into_tensor(self.sqrt_alphas_cumprod, t, x_t.shape) * x_t - + extract_into_tensor(self.sqrt_one_minus_alphas_cumprod, t, x_t.shape) * v + ) + + def predict_eps_from_z_and_v(self, x_t, t, v): + return ( + extract_into_tensor(self.sqrt_alphas_cumprod, t, x_t.shape) * v + + extract_into_tensor(self.sqrt_one_minus_alphas_cumprod, t, x_t.shape) * x_t + ) + + def q_posterior(self, x_start, x_t, t): + posterior_mean = ( + extract_into_tensor(self.posterior_mean_coef1, t, x_t.shape) * x_start + + extract_into_tensor(self.posterior_mean_coef2, t, x_t.shape) * x_t + ) + posterior_variance = extract_into_tensor(self.posterior_variance, t, x_t.shape) + posterior_log_variance_clipped = extract_into_tensor(self.posterior_log_variance_clipped, t, x_t.shape) + return posterior_mean, posterior_variance, posterior_log_variance_clipped + + def p_mean_variance(self, x, t, clip_denoised: bool): + model_out = self.model(x, t) + if self.parameterization == "eps": + x_recon = self.predict_start_from_noise(x, t=t, noise=model_out) + elif self.parameterization == "x0": + x_recon = model_out + if clip_denoised: + x_recon.clamp_(-1., 1.) + + model_mean, posterior_variance, posterior_log_variance = self.q_posterior(x_start=x_recon, x_t=x, t=t) + return model_mean, posterior_variance, posterior_log_variance + + @torch.no_grad() + def p_sample(self, x, t, clip_denoised=True, repeat_noise=False): + b, *_, device = *x.shape, x.device + model_mean, _, model_log_variance = self.p_mean_variance(x=x, t=t, clip_denoised=clip_denoised) + noise = noise_like(x.shape, device, repeat_noise) + # no noise when t == 0 + nonzero_mask = (1 - (t == 0).float()).reshape(b, *((1,) * (len(x.shape) - 1))) + return model_mean + nonzero_mask * (0.5 * model_log_variance).exp() * noise + + @torch.no_grad() + def p_sample_loop(self, shape, return_intermediates=False): + device = self.betas.device + b = shape[0] + img = torch.randn(shape, device=device) + intermediates = [img] + for i in tqdm(reversed(range(0, self.num_timesteps)), desc='Sampling t', total=self.num_timesteps): + img = self.p_sample(img, torch.full((b,), i, device=device, dtype=torch.long), + clip_denoised=self.clip_denoised) + if i % self.log_every_t == 0 or i == self.num_timesteps - 1: + intermediates.append(img) + if return_intermediates: + return img, intermediates + return img + + @torch.no_grad() + def sample(self, batch_size=16, return_intermediates=False): + image_size = self.image_size + channels = self.channels + return self.p_sample_loop((batch_size, channels, image_size, image_size), + return_intermediates=return_intermediates) + + def q_sample(self, x_start, t, noise=None): + noise = default(noise, lambda: torch.randn_like(x_start)) + return (extract_into_tensor(self.sqrt_alphas_cumprod, t, x_start.shape) * x_start + + extract_into_tensor(self.sqrt_one_minus_alphas_cumprod, t, x_start.shape) * noise) + + def get_v(self, x, noise, t): + return ( + extract_into_tensor(self.sqrt_alphas_cumprod, t, x.shape) * noise - + extract_into_tensor(self.sqrt_one_minus_alphas_cumprod, t, x.shape) * x + ) + + def get_loss(self, pred, target, mean=True): + if self.loss_type == 'l1': + loss = (target - pred).abs() + if mean: + loss = loss.mean() + elif self.loss_type == 'l2': + if mean: + loss = torch.nn.functional.mse_loss(target, pred) + else: + loss = torch.nn.functional.mse_loss(target, pred, reduction='none') + else: + raise NotImplementedError("unknown loss type '{loss_type}'") + + return loss + + def p_losses(self, x_start, t, noise=None): + noise = default(noise, lambda: torch.randn_like(x_start)) + x_noisy = self.q_sample(x_start=x_start, t=t, noise=noise) + model_out = self.model(x_noisy, t) + + loss_dict = {} + if self.parameterization == "eps": + target = noise + elif self.parameterization == "x0": + target = x_start + elif self.parameterization == "v": + target = self.get_v(x_start, noise, t) + else: + raise NotImplementedError(f"Parameterization {self.parameterization} not yet supported") + + loss = self.get_loss(model_out, target, mean=False).mean(dim=[1, 2, 3]) + + log_prefix = 'train' if self.training else 'val' + + loss_dict.update({f'{log_prefix}/loss_simple': loss.mean()}) + loss_simple = loss.mean() * self.l_simple_weight + + loss_vlb = (self.lvlb_weights[t] * loss).mean() + loss_dict.update({f'{log_prefix}/loss_vlb': loss_vlb}) + + loss = loss_simple + self.original_elbo_weight * loss_vlb + + loss_dict.update({f'{log_prefix}/loss': loss}) + + return loss, loss_dict + + def forward(self, x, *args, **kwargs): + # b, c, h, w, device, img_size, = *x.shape, x.device, self.image_size + # assert h == img_size and w == img_size, f'height and width of image must be {img_size}' + t = torch.randint(0, self.num_timesteps, (x.shape[0],), device=self.device).long() + return self.p_losses(x, t, *args, **kwargs) + + def get_input(self, batch, k): + x = batch[k] + if len(x.shape) == 3: + x = x[..., None] + x = rearrange(x, 'b h w c -> b c h w') + x = x.to(memory_format=torch.contiguous_format).float() + return x + + def shared_step(self, batch): + x = self.get_input(batch, self.first_stage_key) + loss, loss_dict = self(x) + return loss, loss_dict + + def training_step(self, batch, batch_idx): + for k in self.ucg_training: + p = self.ucg_training[k]["p"] + val = self.ucg_training[k]["val"] + if val is None: + val = "" + for i in range(len(batch[k])): + if self.ucg_prng.choice(2, p=[1 - p, p]): + batch[k][i] = val + + loss, loss_dict = self.shared_step(batch) + + self.log_dict(loss_dict, prog_bar=True, + logger=True, on_step=True, on_epoch=True) + + self.log("global_step", self.global_step, + prog_bar=True, logger=True, on_step=True, on_epoch=False) + + if self.use_scheduler: + lr = self.optimizers().param_groups[0]['lr'] + self.log('lr_abs', lr, prog_bar=True, logger=True, on_step=True, on_epoch=False) + + return loss + + @torch.no_grad() + def validation_step(self, batch, batch_idx): + _, loss_dict_no_ema = self.shared_step(batch) + with self.ema_scope(): + _, loss_dict_ema = self.shared_step(batch) + loss_dict_ema = {key + '_ema': loss_dict_ema[key] for key in loss_dict_ema} + self.log_dict(loss_dict_no_ema, prog_bar=False, logger=True, on_step=False, on_epoch=True) + self.log_dict(loss_dict_ema, prog_bar=False, logger=True, on_step=False, on_epoch=True) + + def on_train_batch_end(self, *args, **kwargs): + if self.use_ema: + self.model_ema(self.model) + + def _get_rows_from_list(self, samples): + n_imgs_per_row = len(samples) + denoise_grid = rearrange(samples, 'n b c h w -> b n c h w') + denoise_grid = rearrange(denoise_grid, 'b n c h w -> (b n) c h w') + denoise_grid = make_grid(denoise_grid, nrow=n_imgs_per_row) + return denoise_grid + + @torch.no_grad() + def log_images(self, batch, N=8, n_row=2, sample=True, return_keys=None, **kwargs): + log = dict() + x = self.get_input(batch, self.first_stage_key) + N = min(x.shape[0], N) + n_row = min(x.shape[0], n_row) + x = x.to(self.device)[:N] + log["inputs"] = x + + # get diffusion row + diffusion_row = list() + x_start = x[:n_row] + + for t in range(self.num_timesteps): + if t % self.log_every_t == 0 or t == self.num_timesteps - 1: + t = repeat(torch.tensor([t]), '1 -> b', b=n_row) + t = t.to(self.device).long() + noise = torch.randn_like(x_start) + x_noisy = self.q_sample(x_start=x_start, t=t, noise=noise) + diffusion_row.append(x_noisy) + + log["diffusion_row"] = self._get_rows_from_list(diffusion_row) + + if sample: + # get denoise row + with self.ema_scope("Plotting"): + samples, denoise_row = self.sample(batch_size=N, return_intermediates=True) + + log["samples"] = samples + log["denoise_row"] = self._get_rows_from_list(denoise_row) + + if return_keys: + if np.intersect1d(list(log.keys()), return_keys).shape[0] == 0: + return log + else: + return {key: log[key] for key in return_keys} + return log + + def configure_optimizers(self): + lr = self.learning_rate + params = list(self.model.parameters()) + if self.learn_logvar: + params = params + [self.logvar] + opt = torch.optim.AdamW(params, lr=lr) + return opt + + +class LatentDiffusion(DDPM): + """main class""" + + def __init__(self, + num_timesteps_cond=None, + cond_stage_key="image", + cond_stage_trainable=False, + concat_mode=True, + cond_stage_forward=None, + conditioning_key=None, + scale_factor=1.0, + scale_by_std=False, + force_null_conditioning=False, + text_enc='custom', + *args, **kwargs): + self.force_null_conditioning = force_null_conditioning + self.num_timesteps_cond = default(num_timesteps_cond, 1) + self.scale_by_std = scale_by_std + assert self.num_timesteps_cond <= kwargs['timesteps'] + # for backwards compatibility after implementation of DiffusionWrapper + if conditioning_key is None: + conditioning_key = 'concat' if concat_mode else 'crossattn' + + ckpt_path = kwargs.pop("ckpt_path", None) + reset_ema = kwargs.pop("reset_ema", False) + reset_num_ema_updates = kwargs.pop("reset_num_ema_updates", False) + ignore_keys = kwargs.pop("ignore_keys", []) + super().__init__(conditioning_key=conditioning_key, *args, **kwargs) + self.concat_mode = concat_mode + self.cond_stage_trainable = cond_stage_trainable + self.cond_stage_key = cond_stage_key + + if not scale_by_std: + self.scale_factor = scale_factor + else: + self.register_buffer('scale_factor', torch.tensor(scale_factor)) + # breakpoint() + self.instantiate_first_stage() + # self.instantiate_cond_stage() + + self.cond_stage_forward = cond_stage_forward + self.clip_denoised = False + self.bbox_tokenizer = None + self.tokenizer=BertTokenizer.from_pretrained('microsoft/BiomedVLP-CXR-BERT-specialized',do_lower_case=True) + self.text_encode_without_mask=False + self.text_transformer=BertModel.from_pretrained("microsoft/BiomedVLP-CXR-BERT-specialized") + self.text_enc=text_enc + + self.restarted_from_ckpt = False + if ckpt_path is not None: + self.init_from_ckpt(ckpt_path, ignore_keys) + self.restarted_from_ckpt = True + if reset_ema: + assert self.use_ema + print( + f"Resetting ema to pure model weights. This is useful when restoring from an ema-only checkpoint.") + self.model_ema = LitEma(self.model) + if reset_num_ema_updates: + print(" +++++++++++ WARNING: RESETTING NUM_EMA UPDATES TO ZERO +++++++++++ ") + assert self.use_ema + self.model_ema.reset_num_updates() + + + def make_cond_schedule(self, ): + self.cond_ids = torch.full(size=(self.num_timesteps,), fill_value=self.num_timesteps - 1, dtype=torch.long) + ids = torch.round(torch.linspace(0, self.num_timesteps - 1, self.num_timesteps_cond)).long() + self.cond_ids[:self.num_timesteps_cond] = ids + + @rank_zero_only + @torch.no_grad() + def on_train_batch_start(self, batch, batch_idx, dataloader_idx): + # only for very first batch + if self.scale_by_std and self.current_epoch == 0 and self.global_step == 0 and batch_idx == 0 and not self.restarted_from_ckpt: + assert self.scale_factor == 1., 'rather not use custom rescaling and std-rescaling simultaneously' + # set rescale weight to 1./std of encodings + # breakpoint() + print("### USING STD-RESCALING ###") + x = super().get_input(batch, self.first_stage_key) + x = x.to(self.device) + encoder_posterior = self.encode_first_stage(x) + z = self.get_first_stage_encoding(encoder_posterior).detach() + del self.scale_factor + self.register_buffer('scale_factor', 1. / z.flatten().std()) + print(f"setting self.scale_factor to {self.scale_factor}") + print("### USING STD-RESCALING ###") + + def register_schedule(self, + given_betas=None, beta_schedule="linear", timesteps=1000, + linear_start=1e-4, linear_end=2e-2, cosine_s=8e-3): + super().register_schedule(given_betas, beta_schedule, timesteps, linear_start, linear_end, cosine_s) + + self.shorten_cond_schedule = self.num_timesteps_cond > 1 + if self.shorten_cond_schedule: + self.make_cond_schedule() + + def instantiate_first_stage(self): + from LeanVAE import LeanVAE + vqgan_ckpt='LeanVAE/ckpts/LeanVAE-dim16.ckpt' + model = LeanVAE.load_from_checkpoint(vqgan_ckpt, strict=False) + self.first_stage_model = model.eval() + self.first_stage_model.train = disabled_train + for param in self.first_stage_model.parameters(): + param.requires_grad = False + + def instantiate_cond_stage(self, config): + # breakpoint() + if not self.cond_stage_trainable: + if config == "__is_first_stage__": + print("Using first stage also as cond stage.") + self.cond_stage_model = self.first_stage_model + elif config == "__is_unconditional__": + print(f"Training {self.__class__.__name__} as an unconditional model.") + self.cond_stage_model = None + # self.be_unconditional = True + else: + model = instantiate_from_config(config) + self.cond_stage_model = model.eval() + self.cond_stage_model.train = disabled_train + for param in self.cond_stage_model.parameters(): + param.requires_grad = False + else: + assert config != '__is_first_stage__' + assert config != '__is_unconditional__' + model = instantiate_from_config(config) + self.cond_stage_model = model + + def _get_denoise_row_from_list(self, samples, desc='', force_no_decoder_quantization=False): + denoise_row = [] + for zd in tqdm(samples, desc=desc): + denoise_row.append(self.decode_first_stage(zd.to(self.device), + force_not_quantize=force_no_decoder_quantization)) + n_imgs_per_row = len(denoise_row) + denoise_row = torch.stack(denoise_row) # n_log_step, n_row, C, H, W + denoise_grid = rearrange(denoise_row, 'n b c h w -> b n c h w') + denoise_grid = rearrange(denoise_grid, 'b n c h w -> (b n) c h w') + denoise_grid = make_grid(denoise_grid, nrow=n_imgs_per_row) + return denoise_grid + + def get_first_stage_encoding(self, encoder_posterior): + # breakpoint() + if isinstance(encoder_posterior, DiagonalGaussianDistribution): + z = encoder_posterior.sample() + elif isinstance(encoder_posterior, torch.Tensor): + z = encoder_posterior + else: + raise NotImplementedError(f"encoder_posterior of type '{type(encoder_posterior)}' not yet implemented") + return self.scale_factor * z + + def get_learned_conditioning(self, c): + if self.cond_stage_forward is None: + if hasattr(self.cond_stage_model, 'encode') and callable(self.cond_stage_model.encode): + c = self.cond_stage_model.encode(c) + if isinstance(c, DiagonalGaussianDistribution): + c = c.mode() + else: + c = self.cond_stage_model(c) + else: + assert hasattr(self.cond_stage_model, self.cond_stage_forward) + c = getattr(self.cond_stage_model, self.cond_stage_forward)(c) + return c + + def meshgrid(self, h, w): + y = torch.arange(0, h).view(h, 1, 1).repeat(1, w, 1) + x = torch.arange(0, w).view(1, w, 1).repeat(h, 1, 1) + + arr = torch.cat([y, x], dim=-1) + return arr + + def delta_border(self, h, w): + """ + :param h: height + :param w: width + :return: normalized distance to image border, + wtith min distance = 0 at border and max dist = 0.5 at image center + """ + lower_right_corner = torch.tensor([h - 1, w - 1]).view(1, 1, 2) + arr = self.meshgrid(h, w) / lower_right_corner + dist_left_up = torch.min(arr, dim=-1, keepdims=True)[0] + dist_right_down = torch.min(1 - arr, dim=-1, keepdims=True)[0] + edge_dist = torch.min(torch.cat([dist_left_up, dist_right_down], dim=-1), dim=-1)[0] + return edge_dist + + def get_weighting(self, h, w, Ly, Lx, device): + weighting = self.delta_border(h, w) + weighting = torch.clip(weighting, self.split_input_params["clip_min_weight"], + self.split_input_params["clip_max_weight"], ) + weighting = weighting.view(1, h * w, 1).repeat(1, 1, Ly * Lx).to(device) + + if self.split_input_params["tie_braker"]: + L_weighting = self.delta_border(Ly, Lx) + L_weighting = torch.clip(L_weighting, + self.split_input_params["clip_min_tie_weight"], + self.split_input_params["clip_max_tie_weight"]) + + L_weighting = L_weighting.view(1, 1, Ly * Lx).to(device) + weighting = weighting * L_weighting + return weighting + + def get_fold_unfold(self, x, kernel_size, stride, uf=1, df=1): # todo load once not every time, shorten code + """ + :param x: img of size (bs, c, h, w) + :return: n img crops of size (n, bs, c, kernel_size[0], kernel_size[1]) + """ + bs, nc, h, w = x.shape + + # number of crops in image + Ly = (h - kernel_size[0]) // stride[0] + 1 + Lx = (w - kernel_size[1]) // stride[1] + 1 + + if uf == 1 and df == 1: + fold_params = dict(kernel_size=kernel_size, dilation=1, padding=0, stride=stride) + unfold = torch.nn.Unfold(**fold_params) + + fold = torch.nn.Fold(output_size=x.shape[2:], **fold_params) + + weighting = self.get_weighting(kernel_size[0], kernel_size[1], Ly, Lx, x.device).to(x.dtype) + normalization = fold(weighting).view(1, 1, h, w) # normalizes the overlap + weighting = weighting.view((1, 1, kernel_size[0], kernel_size[1], Ly * Lx)) + + elif uf > 1 and df == 1: + fold_params = dict(kernel_size=kernel_size, dilation=1, padding=0, stride=stride) + unfold = torch.nn.Unfold(**fold_params) + + fold_params2 = dict(kernel_size=(kernel_size[0] * uf, kernel_size[0] * uf), + dilation=1, padding=0, + stride=(stride[0] * uf, stride[1] * uf)) + fold = torch.nn.Fold(output_size=(x.shape[2] * uf, x.shape[3] * uf), **fold_params2) + + weighting = self.get_weighting(kernel_size[0] * uf, kernel_size[1] * uf, Ly, Lx, x.device).to(x.dtype) + normalization = fold(weighting).view(1, 1, h * uf, w * uf) # normalizes the overlap + weighting = weighting.view((1, 1, kernel_size[0] * uf, kernel_size[1] * uf, Ly * Lx)) + + elif df > 1 and uf == 1: + fold_params = dict(kernel_size=kernel_size, dilation=1, padding=0, stride=stride) + unfold = torch.nn.Unfold(**fold_params) + + fold_params2 = dict(kernel_size=(kernel_size[0] // df, kernel_size[0] // df), + dilation=1, padding=0, + stride=(stride[0] // df, stride[1] // df)) + fold = torch.nn.Fold(output_size=(x.shape[2] // df, x.shape[3] // df), **fold_params2) + + weighting = self.get_weighting(kernel_size[0] // df, kernel_size[1] // df, Ly, Lx, x.device).to(x.dtype) + normalization = fold(weighting).view(1, 1, h // df, w // df) # normalizes the overlap + weighting = weighting.view((1, 1, kernel_size[0] // df, kernel_size[1] // df, Ly * Lx)) + + else: + raise NotImplementedError + + return fold, unfold, normalization, weighting + + @torch.no_grad() + def get_input(self, batch, k, return_first_stage_outputs=False, force_c_encode=False, + cond_key=None, return_original_cond=False, bs=None, return_x=False): + # breakpoint() + if k != 'volume_data' and k != 'slice_data': + x = super().get_input(batch, k) + elif k == 'volume_data': + x = batch[k] + x = x.repeat(1,3,1,1,1) + x = x.to(memory_format=torch.contiguous_format).float() + + if bs is not None: + x = x[:bs] + x = x.to(self.device) + + encoder_posterior = self.encode_first_stage(x) + z = self.get_first_stage_encoding(encoder_posterior).detach() + z = rearrange(z, 'b c z h w -> (b z) c h w') + # breakpoint() + + if self.model.conditioning_key is not None and not self.force_null_conditioning: + if cond_key is None: + cond_key = self.cond_stage_key + if cond_key != self.first_stage_key: + if cond_key in ['caption', 'coordinates_bbox', "txt", "pos_id"]: + xc = batch[cond_key] + elif cond_key in ['class_label', 'cls']: + xc = batch + elif cond_key == 'volume_seg_and_text': + xc = batch['volume_seg'] + xc = xc.repeat(1,3,1,1,1) + + if bs is not None: + xc = xc[:bs] + xc = self.get_first_stage_encoding(self.encode_first_stage(xc)).detach() + z_len=xc.shape[2] + xc = rearrange(xc, 'b c z h w -> (b z) c h w') + xc = xc.to(memory_format=torch.contiguous_format).float() + + # breakpoint() + text = batch['input_text'] + text = list(text) + text_tokens=self.tokenizer(text, return_tensors="pt", padding="max_length", truncation=True, max_length=512).to(self.device) + text_embeddings = self.text_transformer(text_tokens.input_ids, attention_mask = text_tokens.attention_mask ) + enc_text = text_embeddings[0] + enc_text=enc_text[:,None] + enc_text = enc_text.repeat(1,z_len,1,1) + enc_text = rearrange(enc_text, 'b z l c -> (b z) l c') + xc={'c_concat': xc, 'c_crossattn': enc_text} + # breakpoint() + + elif cond_key == 'ref_and_volume_seg': + xc1 = batch['volume_seg'] + slice_num = xc1.shape[1] + xc1 = rearrange(xc1, 'b z h w c -> (b z) c h w') + xc1 = xc1.repeat(1,3,1,1) + if bs is not None: + xc1 = xc1[:bs] + xc1 = self.get_first_stage_encoding(self.encode_first_stage(xc1)).detach() + xc1 = xc1.to(memory_format=torch.contiguous_format).float() + xc2 = batch['volume_ref'] + xc2 = xc2.repeat(1,slice_num,1,1,1) + xc2 = rearrange(xc2, 'b z h w c -> (b z) c h w') + xc2 = xc2.repeat(1,3,1,1) + if bs is not None: + xc2 = xc2[:bs] + xc2 = self.get_first_stage_encoding(self.encode_first_stage(xc2)).detach() + xc2 = xc2.to(memory_format=torch.contiguous_format).float() + xc = torch.cat([xc1, xc2], dim=1) + else: + xc = super().get_input(batch, cond_key).to(self.device) + else: + xc = x + # breakpoint() + if (not self.cond_stage_trainable or force_c_encode) and k != 'volume_data' and cond_key != 'masked_slice' and cond_key != 'masked_slice': + if isinstance(xc, dict) or isinstance(xc, list): + c = self.get_learned_conditioning(xc) + else: + c = self.get_learned_conditioning(xc.to(self.device)) + else: + c = xc + if bs is not None: + c = c[:bs] + # breakpoint() + if self.use_positional_encodings: + pos_x, pos_y = self.compute_latent_shifts(batch) + ckey = __conditioning_keys__[self.model.conditioning_key] + c = {ckey: c, 'pos_x': pos_x, 'pos_y': pos_y} + + else: + c = None + xc = None + if self.use_positional_encodings: + pos_x, pos_y = self.compute_latent_shifts(batch) + c = {'pos_x': pos_x, 'pos_y': pos_y} + out = [z, c] + if return_first_stage_outputs: + xrec = self.decode_first_stage(z) + out.extend([x, xrec]) + if return_x: + out.extend([x]) + if return_original_cond: + out.append(xc) + return out + + @torch.no_grad() + def decode_first_stage(self, z, predict_cids=False, force_not_quantize=False): + if predict_cids: + if z.dim() == 4: + z = torch.argmax(z.exp(), dim=1).long() + z = self.first_stage_model.quantize.get_codebook_entry(z, shape=None) + z = rearrange(z, 'b h w c -> b c h w').contiguous() + + z = 1. / self.scale_factor * z + return self.first_stage_model.decode(z) + + @torch.no_grad() + def encode_first_stage(self, x): + return self.first_stage_model.encode(x) + + def shared_step(self, batch, **kwargs): + x, c = self.get_input(batch, self.first_stage_key) + loss = self(x, c) + return loss + + def forward(self, x, c, *args, **kwargs): + t = torch.randint(0, self.num_timesteps, (x.shape[0],), device=self.device).long() + if self.model.conditioning_key is not None: + assert c is not None + if self.cond_stage_trainable: + c = self.get_learned_conditioning(c) + if self.shorten_cond_schedule: # TODO: drop this option + tc = self.cond_ids[t].to(self.device) + c = self.q_sample(x_start=c, t=tc, noise=torch.randn_like(c.float())) + return self.p_losses(x, c, t, *args, **kwargs) + + def apply_model(self, x_noisy, t, cond, return_ids=False): + if isinstance(cond, dict): + # hybrid case, cond is expected to be a dict + pass + else: + if not isinstance(cond, list): + cond = [cond] + key = 'c_concat' if self.model.conditioning_key == 'concat' else 'c_crossattn' + cond = {key: cond} + # breakpoint() + x_recon = self.model(x_noisy, t, **cond) + + if isinstance(x_recon, tuple) and not return_ids: + return x_recon[0] + else: + return x_recon + + def _predict_eps_from_xstart(self, x_t, t, pred_xstart): + return (extract_into_tensor(self.sqrt_recip_alphas_cumprod, t, x_t.shape) * x_t - pred_xstart) / \ + extract_into_tensor(self.sqrt_recipm1_alphas_cumprod, t, x_t.shape) + + def _prior_bpd(self, x_start): + """ + Get the prior KL term for the variational lower-bound, measured in + bits-per-dim. + This term can't be optimized, as it only depends on the encoder. + :param x_start: the [N x C x ...] tensor of inputs. + :return: a batch of [N] KL values (in bits), one per batch element. + """ + batch_size = x_start.shape[0] + t = torch.tensor([self.num_timesteps - 1] * batch_size, device=x_start.device) + qt_mean, _, qt_log_variance = self.q_mean_variance(x_start, t) + kl_prior = normal_kl(mean1=qt_mean, logvar1=qt_log_variance, mean2=0.0, logvar2=0.0) + return mean_flat(kl_prior) / np.log(2.0) + + def p_losses(self, x_start, cond, t, noise=None): + noise = default(noise, lambda: torch.randn_like(x_start)) + x_noisy = self.q_sample(x_start=x_start, t=t, noise=noise) + model_output = self.apply_model(x_noisy, t, cond) + + loss_dict = {} + prefix = 'train' if self.training else 'val' + + if self.parameterization == "x0": + target = x_start + elif self.parameterization == "eps": + target = noise + elif self.parameterization == "v": + target = self.get_v(x_start, noise, t) + else: + raise NotImplementedError() + + loss_simple = self.get_loss(model_output, target, mean=False).mean([1, 2, 3]) + loss_dict.update({f'{prefix}/loss_simple': loss_simple.mean()}) + + logvar_t = self.logvar[t].to(self.device) + loss = loss_simple / torch.exp(logvar_t) + logvar_t + # loss = loss_simple / torch.exp(self.logvar) + self.logvar + if self.learn_logvar: + loss_dict.update({f'{prefix}/loss_gamma': loss.mean()}) + loss_dict.update({'logvar': self.logvar.data.mean()}) + + loss = self.l_simple_weight * loss.mean() + + loss_vlb = self.get_loss(model_output, target, mean=False).mean(dim=(1, 2, 3)) + loss_vlb = (self.lvlb_weights[t] * loss_vlb).mean() + loss_dict.update({f'{prefix}/loss_vlb': loss_vlb}) + loss += (self.original_elbo_weight * loss_vlb) + loss_dict.update({f'{prefix}/loss': loss}) + + return loss, loss_dict + + def p_mean_variance(self, x, c, t, clip_denoised: bool, return_codebook_ids=False, quantize_denoised=False, + return_x0=False, score_corrector=None, corrector_kwargs=None): + t_in = t + # breakpoint() + model_out = self.apply_model(x, t_in, c, return_ids=return_codebook_ids) + + if score_corrector is not None: + assert self.parameterization == "eps" + model_out = score_corrector.modify_score(self, model_out, x, t, c, **corrector_kwargs) + + if return_codebook_ids: + model_out, logits = model_out + + if self.parameterization == "eps": + x_recon = self.predict_start_from_noise(x, t=t, noise=model_out) + elif self.parameterization == "x0": + x_recon = model_out + else: + raise NotImplementedError() + + if clip_denoised: + x_recon.clamp_(-1., 1.) + if quantize_denoised: + x_recon, _, [_, _, indices] = self.first_stage_model.quantize(x_recon) + model_mean, posterior_variance, posterior_log_variance = self.q_posterior(x_start=x_recon, x_t=x, t=t) + if return_codebook_ids: + return model_mean, posterior_variance, posterior_log_variance, logits + elif return_x0: + return model_mean, posterior_variance, posterior_log_variance, x_recon + else: + return model_mean, posterior_variance, posterior_log_variance + + @torch.no_grad() + def p_sample(self, x, c, t, clip_denoised=False, repeat_noise=False, + return_codebook_ids=False, quantize_denoised=False, return_x0=False, + temperature=1., noise_dropout=0., score_corrector=None, corrector_kwargs=None): + b, *_, device = *x.shape, x.device + outputs = self.p_mean_variance(x=x, c=c, t=t, clip_denoised=clip_denoised, + return_codebook_ids=return_codebook_ids, + quantize_denoised=quantize_denoised, + return_x0=return_x0, + score_corrector=score_corrector, corrector_kwargs=corrector_kwargs) + if return_codebook_ids: + raise DeprecationWarning("Support dropped.") + model_mean, _, model_log_variance, logits = outputs + elif return_x0: + model_mean, _, model_log_variance, x0 = outputs + else: + model_mean, _, model_log_variance = outputs + + noise = noise_like(x.shape, device, repeat_noise) * temperature + if noise_dropout > 0.: + noise = torch.nn.functional.dropout(noise, p=noise_dropout) + # no noise when t == 0 + nonzero_mask = (1 - (t == 0).float()).reshape(b, *((1,) * (len(x.shape) - 1))) + + if return_codebook_ids: + return model_mean + nonzero_mask * (0.5 * model_log_variance).exp() * noise, logits.argmax(dim=1) + if return_x0: + return model_mean + nonzero_mask * (0.5 * model_log_variance).exp() * noise, x0 + else: + return model_mean + nonzero_mask * (0.5 * model_log_variance).exp() * noise + + @torch.no_grad() + def progressive_denoising(self, cond, shape, verbose=True, callback=None, quantize_denoised=False, + img_callback=None, mask=None, x0=None, temperature=1., noise_dropout=0., + score_corrector=None, corrector_kwargs=None, batch_size=None, x_T=None, start_T=None, + log_every_t=None): + if not log_every_t: + log_every_t = self.log_every_t + timesteps = self.num_timesteps + if batch_size is not None: + b = batch_size if batch_size is not None else shape[0] + shape = [batch_size] + list(shape) + else: + b = batch_size = shape[0] + if x_T is None: + img = torch.randn(shape, device=self.device) + else: + img = x_T + intermediates = [] + if cond is not None: + if isinstance(cond, dict): + cond = {key: cond[key][:batch_size] if not isinstance(cond[key], list) else + list(map(lambda x: x[:batch_size], cond[key])) for key in cond} + else: + cond = [c[:batch_size] for c in cond] if isinstance(cond, list) else cond[:batch_size] + + if start_T is not None: + timesteps = min(timesteps, start_T) + iterator = tqdm(reversed(range(0, timesteps)), desc='Progressive Generation', + total=timesteps) if verbose else reversed( + range(0, timesteps)) + if type(temperature) == float: + temperature = [temperature] * timesteps + + for i in iterator: + ts = torch.full((b,), i, device=self.device, dtype=torch.long) + if self.shorten_cond_schedule: + assert self.model.conditioning_key != 'hybrid' + tc = self.cond_ids[ts].to(cond.device) + cond = self.q_sample(x_start=cond, t=tc, noise=torch.randn_like(cond)) + + img, x0_partial = self.p_sample(img, cond, ts, + clip_denoised=self.clip_denoised, + quantize_denoised=quantize_denoised, return_x0=True, + temperature=temperature[i], noise_dropout=noise_dropout, + score_corrector=score_corrector, corrector_kwargs=corrector_kwargs) + if mask is not None: + assert x0 is not None + img_orig = self.q_sample(x0, ts) + img = img_orig * mask + (1. - mask) * img + + if i % log_every_t == 0 or i == timesteps - 1: + intermediates.append(x0_partial) + if callback: callback(i) + if img_callback: img_callback(img, i) + return img, intermediates + + @torch.no_grad() + def p_sample_loop(self, cond, shape, return_intermediates=False, + x_T=None, verbose=True, callback=None, timesteps=None, quantize_denoised=False, + mask=None, x0=None, img_callback=None, start_T=None, + log_every_t=None): + + if not log_every_t: + log_every_t = self.log_every_t + device = self.betas.device + b = shape[0] + if x_T is None: + img = torch.randn(shape, device=device) + else: + img = x_T + + intermediates = [img] + if timesteps is None: + timesteps = self.num_timesteps + + if start_T is not None: + timesteps = min(timesteps, start_T) + iterator = tqdm(reversed(range(0, timesteps)), desc='Sampling t', total=timesteps) if verbose else reversed( + range(0, timesteps)) + + if mask is not None: + assert x0 is not None + assert x0.shape[2:3] == mask.shape[2:3] # spatial size has to match + + for i in iterator: + ts = torch.full((b,), i, device=device, dtype=torch.long) + if self.shorten_cond_schedule: + assert self.model.conditioning_key != 'hybrid' + tc = self.cond_ids[ts].to(cond.device) + cond = self.q_sample(x_start=cond, t=tc, noise=torch.randn_like(cond)) + # breakpoint() + img = self.p_sample(img, cond, ts, + clip_denoised=self.clip_denoised, + quantize_denoised=quantize_denoised) + if mask is not None: + img_orig = self.q_sample(x0, ts) + img = img_orig * mask + (1. - mask) * img + + if i % log_every_t == 0 or i == timesteps - 1: + intermediates.append(img) + if callback: callback(i) + if img_callback: img_callback(img, i) + + if return_intermediates: + return img, intermediates + return img + + @torch.no_grad() + def sample(self, cond, batch_size=16, return_intermediates=False, x_T=None, + verbose=True, timesteps=None, quantize_denoised=False, + mask=None, x0=None, shape=None, **kwargs): + if shape is None: + shape = (batch_size, self.channels, self.image_size, self.image_size) + if cond is not None: + if isinstance(cond, dict): + cond = {key: cond[key][:batch_size] if not isinstance(cond[key], list) else + list(map(lambda x: x[:batch_size], cond[key])) for key in cond} + else: + cond = [c[:batch_size] for c in cond] if isinstance(cond, list) else cond[:batch_size] + return self.p_sample_loop(cond, + shape, + return_intermediates=return_intermediates, x_T=x_T, + verbose=verbose, timesteps=timesteps, quantize_denoised=quantize_denoised, + mask=mask, x0=x0) + + @torch.no_grad() + def sample_log(self, cond, batch_size, ddim, ddim_steps, **kwargs): + if ddim: + ddim_sampler = DDIMSampler(self) + # breakpoint() + # if self.model.conditioning_key == 'crossattn' or self.model.conditioning_key == 'hybrid': + # shape = (self.channels, self.image_size, self.image_size) + # else: + # shape = ((self.channels)//2, self.image_size, self.image_size) + # breakpoint() + shape = (16, self.image_size, self.image_size) + samples, intermediates = ddim_sampler.sample(ddim_steps, batch_size, + shape, cond, verbose=False, **kwargs) + + else: + samples, intermediates = self.sample(cond=cond, batch_size=batch_size, + return_intermediates=True, **kwargs) + + return samples, intermediates + + @torch.no_grad() + def get_unconditional_conditioning(self, batch_size, null_label=None): + if null_label is not None: + xc = null_label + if isinstance(xc, ListConfig): + xc = list(xc) + if isinstance(xc, dict) or isinstance(xc, list): + c = self.get_learned_conditioning(xc) + else: + if hasattr(xc, "to"): + xc = xc.to(self.device) + c = self.get_learned_conditioning(xc) + else: + if self.cond_stage_key in ["class_label", "cls"]: + xc = self.cond_stage_model.get_unconditional_conditioning(batch_size, device=self.device) + return self.get_learned_conditioning(xc) + else: + raise NotImplementedError("todo") + if isinstance(c, list): # in case the encoder gives us a list + for i in range(len(c)): + c[i] = repeat(c[i], '1 ... -> b ...', b=batch_size).to(self.device) + else: + c = repeat(c, '1 ... -> b ...', b=batch_size).to(self.device) + return c + + @torch.no_grad() + def log_images(self, batch, N=32, n_row=4, sample=True, ddim_steps=20, ddim_eta=0., return_keys=None, + quantize_denoised=True, inpaint=True, plot_denoise_rows=False, plot_progressive_rows=True, + plot_diffusion_rows=True, unconditional_guidance_scale=1., unconditional_guidance_label=None, + use_ema_scope=True, + **kwargs): + ema_scope = self.ema_scope if use_ema_scope else nullcontext + use_ddim = ddim_steps is not None + + plot_diffusion_rows = False + plot_progressive_rows = False + inpaint = False + + log = dict() + + z, c, x, xrec, xc = self.get_input(batch, self.first_stage_key, + return_first_stage_outputs=True, + force_c_encode=True, + return_original_cond=True, + # bs=N + ) + N = x.shape[0] + # N = min(x.shape[0], N) + n_row = min(x.shape[0], n_row) + # breakpoint() + log["inputs"] = x + log["reconstruction"] = xrec + # if self.model.conditioning_key is not None: + # if hasattr(self.cond_stage_model, "decode"): + # # xc = self.cond_stage_model.decode(c) + # if c.shape[1] == 8: + # xc = self.decode_first_stage(c[:,:4]) + # elif c.shape[1] == 6: + # xc = self.decode_first_stage(c[:, :3]) + # else: + # xc = self.decode_first_stage(c) + # log["conditioning"] = xc + # elif self.cond_stage_key in ["caption", "txt"]: + # xc = log_txt_as_img((x.shape[2], x.shape[3]), batch[self.cond_stage_key], size=x.shape[2] // 25) + # log["conditioning"] = xc + # elif self.cond_stage_key in ['class_label', "cls"]: + # try: + # xc = log_txt_as_img((x.shape[2], x.shape[3]), batch["human_label"], size=x.shape[2] // 25) + # log['conditioning'] = xc + # except KeyError: + # # probably no "human_label" in batch + # pass + # elif isimage(xc): + # log["conditioning"] = xc + # if ismap(xc): + # log["original_conditioning"] = self.to_rgb(xc) + + if plot_diffusion_rows: + # get diffusion row + diffusion_row = list() + z_start = z[:n_row] + for t in range(self.num_timesteps): + if t % self.log_every_t == 0 or t == self.num_timesteps - 1: + t = repeat(torch.tensor([t]), '1 -> b', b=n_row) + t = t.to(self.device).long() + noise = torch.randn_like(z_start) + z_noisy = self.q_sample(x_start=z_start, t=t, noise=noise) + diffusion_row.append(self.decode_first_stage(z_noisy)) + + diffusion_row = torch.stack(diffusion_row) # n_log_step, n_row, C, H, W + diffusion_grid = rearrange(diffusion_row, 'n b c h w -> b n c h w') + diffusion_grid = rearrange(diffusion_grid, 'b n c h w -> (b n) c h w') + diffusion_grid = make_grid(diffusion_grid, nrow=diffusion_row.shape[0]) + log["diffusion_row"] = diffusion_grid + + # breakpoint() + if sample: + # get denoise row + with ema_scope("Sampling"): + # breakpoint() + samples, z_denoise_row = self.sample_log(cond=c, batch_size=N, ddim=use_ddim, + ddim_steps=ddim_steps, eta=ddim_eta) + # samples, z_denoise_row = self.sample(cond=c, batch_size=N, return_intermediates=True) + x_samples = self.decode_first_stage(samples) + log["samples"] = x_samples + if plot_denoise_rows: + denoise_grid = self._get_denoise_row_from_list(z_denoise_row) + log["denoise_row"] = denoise_grid + + if quantize_denoised and not isinstance(self.first_stage_model, AutoencoderKL) and not isinstance( + self.first_stage_model, IdentityFirstStage): + # also display when quantizing x0 while sampling + with ema_scope("Plotting Quantized Denoised"): + samples, z_denoise_row = self.sample_log(cond=c, batch_size=N, ddim=use_ddim, + ddim_steps=ddim_steps, eta=ddim_eta, + quantize_denoised=True) + # samples, z_denoise_row = self.sample(cond=c, batch_size=N, return_intermediates=True, + # quantize_denoised=True) + x_samples = self.decode_first_stage(samples.to(self.device)) + log["samples_x0_quantized"] = x_samples + + if unconditional_guidance_scale > 1.0: + uc = self.get_unconditional_conditioning(N, unconditional_guidance_label) + if self.model.conditioning_key == "crossattn-adm": + uc = {"c_crossattn": [uc], "c_adm": c["c_adm"]} + with ema_scope("Sampling with classifier-free guidance"): + samples_cfg, _ = self.sample_log(cond=c, batch_size=N, ddim=use_ddim, + ddim_steps=ddim_steps, eta=ddim_eta, + unconditional_guidance_scale=unconditional_guidance_scale, + unconditional_conditioning=uc, + ) + x_samples_cfg = self.decode_first_stage(samples_cfg) + log[f"samples_cfg_scale_{unconditional_guidance_scale:.2f}"] = x_samples_cfg + + if inpaint: + # make a simple center square + b, h, w = z.shape[0], z.shape[2], z.shape[3] + mask = torch.ones(N, h, w).to(self.device) + # zeros will be filled in + mask[:, h // 4:3 * h // 4, w // 4:3 * w // 4] = 0. + mask = mask[:, None, ...] + with ema_scope("Plotting Inpaint"): + samples, _ = self.sample_log(cond=c, batch_size=N, ddim=use_ddim, eta=ddim_eta, + ddim_steps=ddim_steps, x0=z[:N], mask=mask) + x_samples = self.decode_first_stage(samples.to(self.device)) + log["samples_inpainting"] = x_samples + log["mask"] = mask + + # outpaint + mask = 1. - mask + with ema_scope("Plotting Outpaint"): + samples, _ = self.sample_log(cond=c, batch_size=N, ddim=use_ddim, eta=ddim_eta, + ddim_steps=ddim_steps, x0=z[:N], mask=mask) + x_samples = self.decode_first_stage(samples.to(self.device)) + log["samples_outpainting"] = x_samples + + if plot_progressive_rows: + with ema_scope("Plotting Progressives"): + img, progressives = self.progressive_denoising(c, + shape=(self.channels, self.image_size, self.image_size), + batch_size=N) + prog_row = self._get_denoise_row_from_list(progressives, desc="Progressive Generation") + log["progressive_row"] = prog_row + + if return_keys: + if np.intersect1d(list(log.keys()), return_keys).shape[0] == 0: + return log + else: + return {key: log[key] for key in return_keys} + return log + + def configure_optimizers(self): + lr = self.learning_rate + params = list(self.model.parameters()) + if self.cond_stage_trainable: + print(f"{self.__class__.__name__}: Also optimizing conditioner params!") + params = params + list(self.cond_stage_model.parameters()) + if self.learn_logvar: + print('Diffusion model optimizing logvar') + params.append(self.logvar) + opt = torch.optim.AdamW(params, lr=lr) + if self.use_scheduler: + assert 'target' in self.scheduler_config + scheduler = instantiate_from_config(self.scheduler_config) + + print("Setting up LambdaLR scheduler...") + scheduler = [ + { + 'scheduler': LambdaLR(opt, lr_lambda=scheduler.schedule), + 'interval': 'step', + 'frequency': 1 + }] + return [opt], scheduler + return opt + + @torch.no_grad() + def to_rgb(self, x): + x = x.float() + if not hasattr(self, "colorize"): + self.colorize = torch.randn(3, x.shape[1], 1, 1).to(x) + x = nn.functional.conv2d(x, weight=self.colorize) + x = 2. * (x - x.min()) / (x.max() - x.min()) - 1. + return x + + +class DiffusionWrapper(pl.LightningModule): + def __init__(self, diff_model_config, conditioning_key): + super().__init__() + self.sequential_cross_attn = diff_model_config.pop("sequential_crossattn", False) + self.diffusion_model = instantiate_from_config(diff_model_config) + self.conditioning_key = conditioning_key + assert self.conditioning_key in [None, 'concat', 'crossattn', 'hybrid', 'adm', 'hybrid-adm', 'crossattn-adm'] + + def forward(self, x, t, c_concat: list = None, c_crossattn: list = None, c_adm=None): + # breakpoint() + if self.conditioning_key is None: + out = self.diffusion_model(x, t) + elif self.conditioning_key == 'concat': + xc = torch.cat([x] + c_concat, dim=1) + out = self.diffusion_model(xc, t) + elif self.conditioning_key == 'crossattn': + # breakpoint() + xc = torch.cat([x] + [c_concat], dim=1) + cc = torch.cat([c_crossattn], 1) + out = self.diffusion_model(xc, t, context=cc) + elif self.conditioning_key == 'hybrid': + xc = torch.cat([x] + c_crossattn, dim=1) + cc = torch.cat(c_crossattn, 1) + out = self.diffusion_model(xc, t, context=cc) + elif self.conditioning_key == 'hybrid-adm': + assert c_adm is not None + xc = torch.cat([x] + c_concat, dim=1) + cc = torch.cat(c_crossattn, 1) + out = self.diffusion_model(xc, t, context=cc, y=c_adm) + elif self.conditioning_key == 'crossattn-adm': + assert c_adm is not None + cc = torch.cat(c_crossattn, 1) + out = self.diffusion_model(x, t, context=cc, y=c_adm) + elif self.conditioning_key == 'adm': + cc = c_crossattn[0] + out = self.diffusion_model(x, t, y=cc) + else: + raise NotImplementedError() + + return out diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm_pseudo3D.py b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm_pseudo3D.py new file mode 100644 index 0000000000000000000000000000000000000000..fb42735f1851b9e50450ea481657fbeddcdaea54 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm_pseudo3D.py @@ -0,0 +1,1463 @@ +""" +wild mixture of +https://github.com/lucidrains/denoising-diffusion-pytorch/blob/7706bdfc6f527f58d33f84b7b522e61e6e3164b3/denoising_diffusion_pytorch/denoising_diffusion_pytorch.py +https://github.com/openai/improved-diffusion/blob/e94489283bb876ac1477d5dd7709bbbd2d9902ce/improved_diffusion/gaussian_diffusion.py +https://github.com/CompVis/taming-transformers +-- merci +""" + +import torch +import torch.nn as nn +import numpy as np +import pytorch_lightning as pl +from torch.optim.lr_scheduler import LambdaLR +from einops import rearrange, repeat +from contextlib import contextmanager, nullcontext +from functools import partial +import itertools +from tqdm import tqdm +from torchvision.utils import make_grid +from pytorch_lightning.utilities.distributed import rank_zero_only +from omegaconf import ListConfig + +from ldm.util import log_txt_as_img, exists, default, ismap, isimage, mean_flat, count_params, instantiate_from_config +from ldm.modules.ema import LitEma +from ldm.modules.distributions.distributions import normal_kl, DiagonalGaussianDistribution +from ldm.models.autoencoder import IdentityFirstStage, AutoencoderKL +from ldm.modules.diffusionmodules.util import make_beta_schedule, extract_into_tensor, noise_like +from ldm.models.diffusion.ddim import DDIMSampler +from transformers import BertTokenizer,BertModel + + +__conditioning_keys__ = {'concat': 'c_concat', + 'crossattn': 'c_crossattn', + 'adm': 'y'} + +weights = [2000 - i for i in range(1000)] +weights = torch.tensor(weights, dtype=torch.float) + +def disabled_train(self, mode=True): + """Overwrite model.train with this function to make sure train/eval mode + does not change anymore.""" + return self + + +def uniform_on_device(r1, r2, shape, device): + return (r1 - r2) * torch.rand(*shape, device=device) + r2 + + +class DDPM(pl.LightningModule): + # classic DDPM with Gaussian diffusion, in image space + def __init__(self, + unet_config, + timesteps=1000, + beta_schedule="linear", + loss_type="l2", + ckpt_path=None, + ignore_keys=[], + load_only_unet=False, + monitor="val/loss", + use_ema=True, + first_stage_key="image", + image_size=256, + channels=3, + log_every_t=100, + clip_denoised=True, + linear_start=1e-4, + linear_end=2e-2, + cosine_s=8e-3, + given_betas=None, + original_elbo_weight=0., + v_posterior=0., # weight for choosing posterior variance as sigma = (1-v) * beta_tilde + v * beta + l_simple_weight=1., + conditioning_key=None, + parameterization="eps", # all assuming fixed variance schedules + scheduler_config=None, + use_positional_encodings=False, + learn_logvar=False, + logvar_init=0., + make_it_fit=False, + ucg_training=None, + reset_ema=False, + reset_num_ema_updates=False, + ): + super().__init__() + assert parameterization in ["eps", "x0", "v"], 'currently only supporting "eps" and "x0" and "v"' + self.parameterization = parameterization + print(f"{self.__class__.__name__}: Running in {self.parameterization}-prediction mode") + self.cond_stage_model = None + self.clip_denoised = clip_denoised + self.log_every_t = log_every_t + self.first_stage_key = first_stage_key + self.image_size = image_size # try conv? + self.channels = channels + self.use_positional_encodings = use_positional_encodings + self.model = DiffusionWrapper(unet_config, conditioning_key) + count_params(self.model, verbose=True) + self.use_ema = use_ema + if self.use_ema: + self.model_ema = LitEma(self.model) + print(f"Keeping EMAs of {len(list(self.model_ema.buffers()))}.") + + self.use_scheduler = scheduler_config is not None + if self.use_scheduler: + self.scheduler_config = scheduler_config + + self.v_posterior = v_posterior + self.original_elbo_weight = original_elbo_weight + self.l_simple_weight = l_simple_weight + + if monitor is not None: + self.monitor = monitor + self.make_it_fit = make_it_fit + if reset_ema: assert exists(ckpt_path) + if ckpt_path is not None: + self.init_from_ckpt(ckpt_path, ignore_keys=ignore_keys, only_model=load_only_unet) + if reset_ema: + assert self.use_ema + print(f"Resetting ema to pure model weights. This is useful when restoring from an ema-only checkpoint.") + self.model_ema = LitEma(self.model) + if reset_num_ema_updates: + print(" +++++++++++ WARNING: RESETTING NUM_EMA UPDATES TO ZERO +++++++++++ ") + assert self.use_ema + self.model_ema.reset_num_updates() + + self.register_schedule(given_betas=given_betas, beta_schedule=beta_schedule, timesteps=timesteps, + linear_start=linear_start, linear_end=linear_end, cosine_s=cosine_s) + + self.loss_type = loss_type + + self.learn_logvar = learn_logvar + logvar = torch.full(fill_value=logvar_init, size=(self.num_timesteps,)) + if self.learn_logvar: + self.logvar = nn.Parameter(self.logvar, requires_grad=True) + else: + self.register_buffer('logvar', logvar) + + self.ucg_training = ucg_training or dict() + if self.ucg_training: + self.ucg_prng = np.random.RandomState() + + def register_schedule(self, given_betas=None, beta_schedule="linear", timesteps=1000, + linear_start=1e-4, linear_end=2e-2, cosine_s=8e-3): + if exists(given_betas): + betas = given_betas + else: + betas = make_beta_schedule(beta_schedule, timesteps, linear_start=linear_start, linear_end=linear_end, + cosine_s=cosine_s) + alphas = 1. - betas + alphas_cumprod = np.cumprod(alphas, axis=0) + alphas_cumprod_prev = np.append(1., alphas_cumprod[:-1]) + + timesteps, = betas.shape + self.num_timesteps = int(timesteps) + self.linear_start = linear_start + self.linear_end = linear_end + assert alphas_cumprod.shape[0] == self.num_timesteps, 'alphas have to be defined for each timestep' + + to_torch = partial(torch.tensor, dtype=torch.float32) + + self.register_buffer('betas', to_torch(betas)) + self.register_buffer('alphas_cumprod', to_torch(alphas_cumprod)) + self.register_buffer('alphas_cumprod_prev', to_torch(alphas_cumprod_prev)) + + # calculations for diffusion q(x_t | x_{t-1}) and others + self.register_buffer('sqrt_alphas_cumprod', to_torch(np.sqrt(alphas_cumprod))) + self.register_buffer('sqrt_one_minus_alphas_cumprod', to_torch(np.sqrt(1. - alphas_cumprod))) + self.register_buffer('log_one_minus_alphas_cumprod', to_torch(np.log(1. - alphas_cumprod))) + self.register_buffer('sqrt_recip_alphas_cumprod', to_torch(np.sqrt(1. / alphas_cumprod))) + self.register_buffer('sqrt_recipm1_alphas_cumprod', to_torch(np.sqrt(1. / alphas_cumprod - 1))) + + # calculations for posterior q(x_{t-1} | x_t, x_0) + posterior_variance = (1 - self.v_posterior) * betas * (1. - alphas_cumprod_prev) / ( + 1. - alphas_cumprod) + self.v_posterior * betas + # above: equal to 1. / (1. / (1. - alpha_cumprod_tm1) + alpha_t / beta_t) + self.register_buffer('posterior_variance', to_torch(posterior_variance)) + # below: log calculation clipped because the posterior variance is 0 at the beginning of the diffusion chain + self.register_buffer('posterior_log_variance_clipped', to_torch(np.log(np.maximum(posterior_variance, 1e-20)))) + self.register_buffer('posterior_mean_coef1', to_torch( + betas * np.sqrt(alphas_cumprod_prev) / (1. - alphas_cumprod))) + self.register_buffer('posterior_mean_coef2', to_torch( + (1. - alphas_cumprod_prev) * np.sqrt(alphas) / (1. - alphas_cumprod))) + + if self.parameterization == "eps": + lvlb_weights = self.betas ** 2 / ( + 2 * self.posterior_variance * to_torch(alphas) * (1 - self.alphas_cumprod)) + elif self.parameterization == "x0": + lvlb_weights = 0.5 * np.sqrt(torch.Tensor(alphas_cumprod)) / (2. * 1 - torch.Tensor(alphas_cumprod)) + elif self.parameterization == "v": + lvlb_weights = torch.ones_like(self.betas ** 2 / ( + 2 * self.posterior_variance * to_torch(alphas) * (1 - self.alphas_cumprod))) + else: + raise NotImplementedError("mu not supported") + lvlb_weights[0] = lvlb_weights[1] + self.register_buffer('lvlb_weights', lvlb_weights, persistent=False) + assert not torch.isnan(self.lvlb_weights).all() + + @contextmanager + def ema_scope(self, context=None): + if self.use_ema: + self.model_ema.store(self.model.parameters()) + self.model_ema.copy_to(self.model) + if context is not None: + print(f"{context}: Switched to EMA weights") + try: + yield None + finally: + if self.use_ema: + self.model_ema.restore(self.model.parameters()) + if context is not None: + print(f"{context}: Restored training weights") + + @torch.no_grad() + def init_from_ckpt(self, path, ignore_keys=list(), only_model=False): + sd = torch.load(path, map_location="cpu") + if "state_dict" in list(sd.keys()): + sd = sd["state_dict"] + keys = list(sd.keys()) + for k in keys: + for ik in ignore_keys: + if k.startswith(ik): + print("Deleting key {} from state_dict.".format(k)) + del sd[k] + if self.make_it_fit: + n_params = len([name for name, _ in + itertools.chain(self.named_parameters(), + self.named_buffers())]) + for name, param in tqdm( + itertools.chain(self.named_parameters(), + self.named_buffers()), + desc="Fitting old weights to new weights", + total=n_params + ): + if not name in sd: + continue + old_shape = sd[name].shape + new_shape = param.shape + assert len(old_shape) == len(new_shape) + if len(new_shape) > 2: + # we only modify first two axes + assert new_shape[2:] == old_shape[2:] + # assumes first axis corresponds to output dim + if not new_shape == old_shape: + new_param = param.clone() + old_param = sd[name] + if len(new_shape) == 1: + for i in range(new_param.shape[0]): + new_param[i] = old_param[i % old_shape[0]] + elif len(new_shape) >= 2: + for i in range(new_param.shape[0]): + for j in range(new_param.shape[1]): + new_param[i, j] = old_param[i % old_shape[0], j % old_shape[1]] + + n_used_old = torch.ones(old_shape[1]) + for j in range(new_param.shape[1]): + n_used_old[j % old_shape[1]] += 1 + n_used_new = torch.zeros(new_shape[1]) + for j in range(new_param.shape[1]): + n_used_new[j] = n_used_old[j % old_shape[1]] + + n_used_new = n_used_new[None, :] + while len(n_used_new.shape) < len(new_shape): + n_used_new = n_used_new.unsqueeze(-1) + new_param /= n_used_new + + sd[name] = new_param + + missing, unexpected = self.load_state_dict(sd, strict=False) if not only_model else self.model.load_state_dict( + sd, strict=False) + print(f"Restored from {path} with {len(missing)} missing and {len(unexpected)} unexpected keys") + if len(missing) > 0: + print(f"Missing Keys:\n {missing}") + if len(unexpected) > 0: + print(f"\nUnexpected Keys:\n {unexpected}") + + def q_mean_variance(self, x_start, t): + """ + Get the distribution q(x_t | x_0). + :param x_start: the [N x C x ...] tensor of noiseless inputs. + :param t: the number of diffusion steps (minus 1). Here, 0 means one step. + :return: A tuple (mean, variance, log_variance), all of x_start's shape. + """ + mean = (extract_into_tensor(self.sqrt_alphas_cumprod, t, x_start.shape) * x_start) + variance = extract_into_tensor(1.0 - self.alphas_cumprod, t, x_start.shape) + log_variance = extract_into_tensor(self.log_one_minus_alphas_cumprod, t, x_start.shape) + return mean, variance, log_variance + + def predict_start_from_noise(self, x_t, t, noise): + return ( + extract_into_tensor(self.sqrt_recip_alphas_cumprod, t, x_t.shape) * x_t - + extract_into_tensor(self.sqrt_recipm1_alphas_cumprod, t, x_t.shape) * noise + ) + + def predict_start_from_z_and_v(self, x_t, t, v): + # self.register_buffer('sqrt_alphas_cumprod', to_torch(np.sqrt(alphas_cumprod))) + # self.register_buffer('sqrt_one_minus_alphas_cumprod', to_torch(np.sqrt(1. - alphas_cumprod))) + return ( + extract_into_tensor(self.sqrt_alphas_cumprod, t, x_t.shape) * x_t - + extract_into_tensor(self.sqrt_one_minus_alphas_cumprod, t, x_t.shape) * v + ) + + def predict_eps_from_z_and_v(self, x_t, t, v): + return ( + extract_into_tensor(self.sqrt_alphas_cumprod, t, x_t.shape) * v + + extract_into_tensor(self.sqrt_one_minus_alphas_cumprod, t, x_t.shape) * x_t + ) + + def q_posterior(self, x_start, x_t, t): + posterior_mean = ( + extract_into_tensor(self.posterior_mean_coef1, t, x_t.shape) * x_start + + extract_into_tensor(self.posterior_mean_coef2, t, x_t.shape) * x_t + ) + posterior_variance = extract_into_tensor(self.posterior_variance, t, x_t.shape) + posterior_log_variance_clipped = extract_into_tensor(self.posterior_log_variance_clipped, t, x_t.shape) + return posterior_mean, posterior_variance, posterior_log_variance_clipped + + def p_mean_variance(self, x, t, clip_denoised: bool): + model_out = self.model(x, t) + if self.parameterization == "eps": + x_recon = self.predict_start_from_noise(x, t=t, noise=model_out) + elif self.parameterization == "x0": + x_recon = model_out + if clip_denoised: + x_recon.clamp_(-1., 1.) + + model_mean, posterior_variance, posterior_log_variance = self.q_posterior(x_start=x_recon, x_t=x, t=t) + return model_mean, posterior_variance, posterior_log_variance + + @torch.no_grad() + def p_sample(self, x, t, clip_denoised=True, repeat_noise=False): + b, *_, device = *x.shape, x.device + model_mean, _, model_log_variance = self.p_mean_variance(x=x, t=t, clip_denoised=clip_denoised) + noise = noise_like(x.shape, device, repeat_noise) + # no noise when t == 0 + nonzero_mask = (1 - (t == 0).float()).reshape(b, *((1,) * (len(x.shape) - 1))) + return model_mean + nonzero_mask * (0.5 * model_log_variance).exp() * noise + + @torch.no_grad() + def p_sample_loop(self, shape, return_intermediates=False): + device = self.betas.device + b = shape[0] + img = torch.randn(shape, device=device) + intermediates = [img] + for i in tqdm(reversed(range(0, self.num_timesteps)), desc='Sampling t', total=self.num_timesteps): + img = self.p_sample(img, torch.full((b,), i, device=device, dtype=torch.long), + clip_denoised=self.clip_denoised) + if i % self.log_every_t == 0 or i == self.num_timesteps - 1: + intermediates.append(img) + if return_intermediates: + return img, intermediates + return img + + @torch.no_grad() + def sample(self, batch_size=16, return_intermediates=False): + image_size = self.image_size + channels = self.channels + return self.p_sample_loop((batch_size, channels, image_size, image_size), + return_intermediates=return_intermediates) + + def q_sample(self, x_start, t, noise=None): + noise = default(noise, lambda: torch.randn_like(x_start)) + # breakpoint() + return (extract_into_tensor(self.sqrt_alphas_cumprod, t, x_start.shape) * x_start + + extract_into_tensor(self.sqrt_one_minus_alphas_cumprod, t, x_start.shape) * noise) + + def get_v(self, x, noise, t): + return ( + extract_into_tensor(self.sqrt_alphas_cumprod, t, x.shape) * noise - + extract_into_tensor(self.sqrt_one_minus_alphas_cumprod, t, x.shape) * x + ) + + def get_loss(self, pred, target, mean=True): + if self.loss_type == 'l1': + loss = (target - pred).abs() + if mean: + loss = loss.mean() + elif self.loss_type == 'l2': + if mean: + loss = torch.nn.functional.mse_loss(target, pred) + else: + loss = torch.nn.functional.mse_loss(target, pred, reduction='none') + else: + raise NotImplementedError("unknown loss type '{loss_type}'") + + return loss + + def p_losses(self, x_start, t, noise=None): + noise = default(noise, lambda: torch.randn_like(x_start)) + x_noisy = self.q_sample(x_start=x_start, t=t, noise=noise) + model_out = self.model(x_noisy, t) + + loss_dict = {} + if self.parameterization == "eps": + target = noise + elif self.parameterization == "x0": + target = x_start + elif self.parameterization == "v": + target = self.get_v(x_start, noise, t) + else: + raise NotImplementedError(f"Parameterization {self.parameterization} not yet supported") + + loss = self.get_loss(model_out, target, mean=False).mean(dim=[1, 2, 3]) + + log_prefix = 'train' if self.training else 'val' + + loss_dict.update({f'{log_prefix}/loss_simple': loss.mean()}) + loss_simple = loss.mean() * self.l_simple_weight + + loss_vlb = (self.lvlb_weights[t] * loss).mean() + loss_dict.update({f'{log_prefix}/loss_vlb': loss_vlb}) + + loss = loss_simple + self.original_elbo_weight * loss_vlb + + loss_dict.update({f'{log_prefix}/loss': loss}) + + return loss, loss_dict + + def forward(self, x, *args, **kwargs): + # b, c, h, w, device, img_size, = *x.shape, x.device, self.image_size + # assert h == img_size and w == img_size, f'height and width of image must be {img_size}' + t = torch.randint(0, self.num_timesteps, (x.shape[0],), device=self.device).long() + return self.p_losses(x, t, *args, **kwargs) + + def get_input(self, batch, k): + x = batch[k] + if len(x.shape) == 3: + x = x[..., None] + x = rearrange(x, 'b h w c -> b c h w') + x = x.to(memory_format=torch.contiguous_format).float() + return x + + def shared_step(self, batch): + x = self.get_input(batch, self.first_stage_key) + loss, loss_dict = self(x) + return loss, loss_dict + + def training_step(self, batch, batch_idx): + for k in self.ucg_training: + p = self.ucg_training[k]["p"] + val = self.ucg_training[k]["val"] + if val is None: + val = "" + for i in range(len(batch[k])): + if self.ucg_prng.choice(2, p=[1 - p, p]): + batch[k][i] = val + + loss, loss_dict = self.shared_step(batch) + + self.log_dict(loss_dict, prog_bar=True, + logger=True, on_step=True, on_epoch=True) + + self.log("global_step", self.global_step, + prog_bar=True, logger=True, on_step=True, on_epoch=False) + + if self.use_scheduler: + lr = self.optimizers().param_groups[0]['lr'] + self.log('lr_abs', lr, prog_bar=True, logger=True, on_step=True, on_epoch=False) + + return loss + + @torch.no_grad() + def validation_step(self, batch, batch_idx): + _, loss_dict_no_ema = self.shared_step(batch) + with self.ema_scope(): + _, loss_dict_ema = self.shared_step(batch) + loss_dict_ema = {key + '_ema': loss_dict_ema[key] for key in loss_dict_ema} + self.log_dict(loss_dict_no_ema, prog_bar=False, logger=True, on_step=False, on_epoch=True) + self.log_dict(loss_dict_ema, prog_bar=False, logger=True, on_step=False, on_epoch=True) + + def on_train_batch_end(self, *args, **kwargs): + if self.use_ema: + self.model_ema(self.model) + + def _get_rows_from_list(self, samples): + n_imgs_per_row = len(samples) + denoise_grid = rearrange(samples, 'n b c h w -> b n c h w') + denoise_grid = rearrange(denoise_grid, 'b n c h w -> (b n) c h w') + denoise_grid = make_grid(denoise_grid, nrow=n_imgs_per_row) + return denoise_grid + + @torch.no_grad() + def log_images(self, batch, N=8, n_row=2, sample=True, return_keys=None, **kwargs): + log = dict() + x = self.get_input(batch, self.first_stage_key) + N = min(x.shape[0], N) + n_row = min(x.shape[0], n_row) + x = x.to(self.device)[:N] + log["inputs"] = x + + # get diffusion row + diffusion_row = list() + x_start = x[:n_row] + + for t in range(self.num_timesteps): + if t % self.log_every_t == 0 or t == self.num_timesteps - 1: + t = repeat(torch.tensor([t]), '1 -> b', b=n_row) + t = t.to(self.device).long() + noise = torch.randn_like(x_start) + x_noisy = self.q_sample(x_start=x_start, t=t, noise=noise) + diffusion_row.append(x_noisy) + + log["diffusion_row"] = self._get_rows_from_list(diffusion_row) + + if sample: + # get denoise row + with self.ema_scope("Plotting"): + samples, denoise_row = self.sample(batch_size=N, return_intermediates=True) + + log["samples"] = samples + log["denoise_row"] = self._get_rows_from_list(denoise_row) + + if return_keys: + if np.intersect1d(list(log.keys()), return_keys).shape[0] == 0: + return log + else: + return {key: log[key] for key in return_keys} + return log + + def configure_optimizers(self): + lr = self.learning_rate + params = list(self.model.parameters()) + if self.learn_logvar: + params = params + [self.logvar] + opt = torch.optim.AdamW(params, lr=lr) + return opt + + +class LatentDiffusion(DDPM): + """main class""" + + def __init__(self, + num_timesteps_cond=None, + cond_stage_key="image", + cond_stage_trainable=False, + concat_mode=True, + cond_stage_forward=None, + conditioning_key=None, + scale_factor=1.0, + scale_by_std=False, + force_null_conditioning=False, + fix_t=False, + text_enc='custom', + *args, **kwargs): + self.force_null_conditioning = force_null_conditioning + self.num_timesteps_cond = default(num_timesteps_cond, 1) + self.scale_by_std = scale_by_std + assert self.num_timesteps_cond <= kwargs['timesteps'] + # for backwards compatibility after implementation of DiffusionWrapper + if conditioning_key is None: + conditioning_key = 'concat' if concat_mode else 'crossattn' + + ckpt_path = kwargs.pop("ckpt_path", None) + reset_ema = kwargs.pop("reset_ema", False) + reset_num_ema_updates = kwargs.pop("reset_num_ema_updates", False) + ignore_keys = kwargs.pop("ignore_keys", []) + super().__init__(conditioning_key=conditioning_key, *args, **kwargs) + self.concat_mode = concat_mode + self.cond_stage_trainable = cond_stage_trainable + self.cond_stage_key = cond_stage_key + + if not scale_by_std: + self.scale_factor = scale_factor + else: + self.register_buffer('scale_factor', torch.tensor(scale_factor)) + self.instantiate_first_stage() + # self.instantiate_cond_stage() + self.cond_stage_forward = cond_stage_forward + self.clip_denoised = False + self.bbox_tokenizer = None + self.tokenizer=BertTokenizer.from_pretrained('microsoft/BiomedVLP-CXR-BERT-specialized',do_lower_case=True) + self.text_encode_without_mask=False + self.text_transformer=BertModel.from_pretrained("microsoft/BiomedVLP-CXR-BERT-specialized") + + self.restarted_from_ckpt = False + if ckpt_path is not None: + self.init_from_ckpt(ckpt_path, ignore_keys) + self.restarted_from_ckpt = True + if reset_ema: + assert self.use_ema + print( + f"Resetting ema to pure model weights. This is useful when restoring from an ema-only checkpoint.") + self.model_ema = LitEma(self.model) + if reset_num_ema_updates: + print(" +++++++++++ WARNING: RESETTING NUM_EMA UPDATES TO ZERO +++++++++++ ") + assert self.use_ema + self.model_ema.reset_num_updates() + + self.fix_t = fix_t + + def make_cond_schedule(self, ): + self.cond_ids = torch.full(size=(self.num_timesteps,), fill_value=self.num_timesteps - 1, dtype=torch.long) + ids = torch.round(torch.linspace(0, self.num_timesteps - 1, self.num_timesteps_cond)).long() + self.cond_ids[:self.num_timesteps_cond] = ids + + @rank_zero_only + @torch.no_grad() + def on_train_batch_start(self, batch, batch_idx, dataloader_idx): + # only for very first batch + if self.scale_by_std and self.current_epoch == 0 and self.global_step == 0 and batch_idx == 0 and not self.restarted_from_ckpt: + assert self.scale_factor == 1., 'rather not use custom rescaling and std-rescaling simultaneously' + # set rescale weight to 1./std of encodings + print("### USING STD-RESCALING ###") + x = super().get_input(batch, self.first_stage_key) + x = x.to(self.device) + encoder_posterior = self.encode_first_stage(x) + z = self.get_first_stage_encoding(encoder_posterior).detach() + del self.scale_factor + self.register_buffer('scale_factor', 1. / z.flatten().std()) + print(f"setting self.scale_factor to {self.scale_factor}") + print("### USING STD-RESCALING ###") + + def register_schedule(self, + given_betas=None, beta_schedule="linear", timesteps=1000, + linear_start=1e-4, linear_end=2e-2, cosine_s=8e-3): + super().register_schedule(given_betas, beta_schedule, timesteps, linear_start, linear_end, cosine_s) + + self.shorten_cond_schedule = self.num_timesteps_cond > 1 + if self.shorten_cond_schedule: + self.make_cond_schedule() + + def instantiate_first_stage(self): + from LeanVAE import LeanVAE + vqgan_ckpt='LeanVAE/ckpts/LeanVAE-dim16.ckpt' + model = LeanVAE.load_from_checkpoint(vqgan_ckpt, strict=False) + self.first_stage_model = model.eval() + self.first_stage_model.train = disabled_train + for param in self.first_stage_model.parameters(): + param.requires_grad = False + + def instantiate_cond_stage(self, config): + if not self.cond_stage_trainable: + if config == "__is_first_stage__": + print("Using first stage also as cond stage.") + self.cond_stage_model = self.first_stage_model + elif config == "__is_unconditional__": + print(f"Training {self.__class__.__name__} as an unconditional model.") + self.cond_stage_model = None + # self.be_unconditional = True + else: + model = instantiate_from_config(config) + self.cond_stage_model = model.eval() + self.cond_stage_model.train = disabled_train + for param in self.cond_stage_model.parameters(): + param.requires_grad = False + else: + assert config != '__is_first_stage__' + assert config != '__is_unconditional__' + model = instantiate_from_config(config) + self.cond_stage_model = model + + def _get_denoise_row_from_list(self, samples, desc='', force_no_decoder_quantization=False): + denoise_row = [] + for zd in tqdm(samples, desc=desc): + denoise_row.append(self.decode_first_stage(zd.to(self.device), + force_not_quantize=force_no_decoder_quantization)) + n_imgs_per_row = len(denoise_row) + denoise_row = torch.stack(denoise_row) # n_log_step, n_row, C, H, W + denoise_grid = rearrange(denoise_row, 'n b c h w -> b n c h w') + denoise_grid = rearrange(denoise_grid, 'b n c h w -> (b n) c h w') + denoise_grid = make_grid(denoise_grid, nrow=n_imgs_per_row) + return denoise_grid + + def get_first_stage_encoding(self, encoder_posterior): + if isinstance(encoder_posterior, DiagonalGaussianDistribution): + z = encoder_posterior.sample() + elif isinstance(encoder_posterior, torch.Tensor): + z = encoder_posterior + else: + raise NotImplementedError(f"encoder_posterior of type '{type(encoder_posterior)}' not yet implemented") + return self.scale_factor * z + + def get_learned_conditioning(self, c): + if self.cond_stage_forward is None: + if hasattr(self.cond_stage_model, 'encode') and callable(self.cond_stage_model.encode): + c = self.cond_stage_model.encode(c) + if isinstance(c, DiagonalGaussianDistribution): + c = c.mode() + else: + c = self.cond_stage_model(c) + else: + assert hasattr(self.cond_stage_model, self.cond_stage_forward) + c = getattr(self.cond_stage_model, self.cond_stage_forward)(c) + return c + + def meshgrid(self, h, w): + y = torch.arange(0, h).view(h, 1, 1).repeat(1, w, 1) + x = torch.arange(0, w).view(1, w, 1).repeat(h, 1, 1) + + arr = torch.cat([y, x], dim=-1) + return arr + + def delta_border(self, h, w): + """ + :param h: height + :param w: width + :return: normalized distance to image border, + wtith min distance = 0 at border and max dist = 0.5 at image center + """ + lower_right_corner = torch.tensor([h - 1, w - 1]).view(1, 1, 2) + arr = self.meshgrid(h, w) / lower_right_corner + dist_left_up = torch.min(arr, dim=-1, keepdims=True)[0] + dist_right_down = torch.min(1 - arr, dim=-1, keepdims=True)[0] + edge_dist = torch.min(torch.cat([dist_left_up, dist_right_down], dim=-1), dim=-1)[0] + return edge_dist + + def get_weighting(self, h, w, Ly, Lx, device): + weighting = self.delta_border(h, w) + weighting = torch.clip(weighting, self.split_input_params["clip_min_weight"], + self.split_input_params["clip_max_weight"], ) + weighting = weighting.view(1, h * w, 1).repeat(1, 1, Ly * Lx).to(device) + + if self.split_input_params["tie_braker"]: + L_weighting = self.delta_border(Ly, Lx) + L_weighting = torch.clip(L_weighting, + self.split_input_params["clip_min_tie_weight"], + self.split_input_params["clip_max_tie_weight"]) + + L_weighting = L_weighting.view(1, 1, Ly * Lx).to(device) + weighting = weighting * L_weighting + return weighting + + def get_fold_unfold(self, x, kernel_size, stride, uf=1, df=1): # todo load once not every time, shorten code + """ + :param x: img of size (bs, c, h, w) + :return: n img crops of size (n, bs, c, kernel_size[0], kernel_size[1]) + """ + bs, nc, h, w = x.shape + + # number of crops in image + Ly = (h - kernel_size[0]) // stride[0] + 1 + Lx = (w - kernel_size[1]) // stride[1] + 1 + + if uf == 1 and df == 1: + fold_params = dict(kernel_size=kernel_size, dilation=1, padding=0, stride=stride) + unfold = torch.nn.Unfold(**fold_params) + + fold = torch.nn.Fold(output_size=x.shape[2:], **fold_params) + + weighting = self.get_weighting(kernel_size[0], kernel_size[1], Ly, Lx, x.device).to(x.dtype) + normalization = fold(weighting).view(1, 1, h, w) # normalizes the overlap + weighting = weighting.view((1, 1, kernel_size[0], kernel_size[1], Ly * Lx)) + + elif uf > 1 and df == 1: + fold_params = dict(kernel_size=kernel_size, dilation=1, padding=0, stride=stride) + unfold = torch.nn.Unfold(**fold_params) + + fold_params2 = dict(kernel_size=(kernel_size[0] * uf, kernel_size[0] * uf), + dilation=1, padding=0, + stride=(stride[0] * uf, stride[1] * uf)) + fold = torch.nn.Fold(output_size=(x.shape[2] * uf, x.shape[3] * uf), **fold_params2) + + weighting = self.get_weighting(kernel_size[0] * uf, kernel_size[1] * uf, Ly, Lx, x.device).to(x.dtype) + normalization = fold(weighting).view(1, 1, h * uf, w * uf) # normalizes the overlap + weighting = weighting.view((1, 1, kernel_size[0] * uf, kernel_size[1] * uf, Ly * Lx)) + + elif df > 1 and uf == 1: + fold_params = dict(kernel_size=kernel_size, dilation=1, padding=0, stride=stride) + unfold = torch.nn.Unfold(**fold_params) + + fold_params2 = dict(kernel_size=(kernel_size[0] // df, kernel_size[0] // df), + dilation=1, padding=0, + stride=(stride[0] // df, stride[1] // df)) + fold = torch.nn.Fold(output_size=(x.shape[2] // df, x.shape[3] // df), **fold_params2) + + weighting = self.get_weighting(kernel_size[0] // df, kernel_size[1] // df, Ly, Lx, x.device).to(x.dtype) + normalization = fold(weighting).view(1, 1, h // df, w // df) # normalizes the overlap + weighting = weighting.view((1, 1, kernel_size[0] // df, kernel_size[1] // df, Ly * Lx)) + + else: + raise NotImplementedError + + return fold, unfold, normalization, weighting + + @torch.no_grad() + def get_input(self, batch, k, return_first_stage_outputs=False, force_c_encode=False, + cond_key=None, return_original_cond=False, bs=None, return_x=False): + # breakpoint() + if k != 'volume_data': + x = super().get_input(batch, k) + else: + x = batch[k] + x = x.repeat(1,3,1,1,1) + x = x.to(memory_format=torch.contiguous_format).float() + if bs is not None: + x = x[:bs] + x = x.to(self.device) + # breakpoint() + encoder_posterior = self.encode_first_stage(x) + z = self.get_first_stage_encoding(encoder_posterior).detach() + z = rearrange(z, 'b c z h w -> (b z) c h w') + + if self.model.conditioning_key is not None and not self.force_null_conditioning: + if cond_key is None: + cond_key = self.cond_stage_key + if cond_key != self.first_stage_key: + if cond_key in ['caption', 'coordinates_bbox', "txt"]: + xc = batch[cond_key] + elif cond_key in ['class_label', 'cls']: + xc = batch + elif cond_key == 'volume_seg': + xc = batch['volume_seg'] + xc = rearrange(xc, 'b z h w c -> (b z) c h w') + xc = xc.repeat(1,3,1,1) + if bs is not None: + xc = xc[:bs] + xc = self.get_first_stage_encoding(self.encode_first_stage(xc)).detach() + xc = xc.to(memory_format=torch.contiguous_format).float() + elif cond_key == 'volume_seg_and_text': + xc = batch['volume_seg'] + xc = xc.repeat(1,3,1,1,1) + + if bs is not None: + xc = xc[:bs] + xc = self.get_first_stage_encoding(self.encode_first_stage(xc)).detach() + z_len=xc.shape[2] + xc = rearrange(xc, 'b c z h w -> (b z) c h w') + xc = xc.to(memory_format=torch.contiguous_format).float() + + # breakpoint() + text = batch['input_text'] + text = list(text) + text_tokens=self.tokenizer(text, return_tensors="pt", padding="max_length", truncation=True, max_length=512).to(self.device) + text_embeddings = self.text_transformer(text_tokens.input_ids, attention_mask = text_tokens.attention_mask ) + enc_text = text_embeddings[0] + enc_text=enc_text[:,None] + enc_text = enc_text.repeat(1,z_len,1,1) + enc_text = rearrange(enc_text, 'b z l c -> (b z) l c') + xc={'c_concat': xc, 'c_crossattn': enc_text} + + elif cond_key == 'ref_and_volume_seg': + xc1 = batch['volume_seg'] + slice_num = xc1.shape[1] + xc1 = rearrange(xc1, 'b z h w c -> (b z) c h w') + xc1 = xc1.repeat(1,3,1,1) + if bs is not None: + xc1 = xc1[:bs] + xc1 = self.get_first_stage_encoding(self.encode_first_stage(xc1)).detach() + xc1 = xc1.to(memory_format=torch.contiguous_format).float() + xc2 = batch['volume_ref'] + xc2 = xc2.repeat(1,slice_num,1,1,1) + xc2 = rearrange(xc2, 'b z h w c -> (b z) c h w') + xc2 = xc2.repeat(1,3,1,1) + if bs is not None: + xc2 = xc2[:bs] + xc2 = self.get_first_stage_encoding(self.encode_first_stage(xc2)).detach() + xc2 = xc2.to(memory_format=torch.contiguous_format).float() + xc = torch.cat([xc1, xc2], dim=1) + elif cond_key == 'masked_volume': + xc = batch['masked_data'] + mask_tumor = batch['tumor_mask'] + # breakpoint() + mask_tumor = rearrange(mask_tumor, 'b z h w c -> (b z) c h w') + xc = rearrange(xc, 'b z h w c -> (b z) c h w') + xc = xc.repeat(1,3,1,1) + if bs is not None: + xc = xc[:bs] + mask_tumor=mask_tumor[:bs] + xc = self.get_first_stage_encoding(self.encode_first_stage(xc)).detach() + # breakpoint() + xc = torch.cat([xc, mask_tumor], dim=1).detach() + xc = xc.to(memory_format=torch.contiguous_format).float() + else: + xc = super().get_input(batch, cond_key).to(self.device) + else: + xc = x + if (not self.cond_stage_trainable or force_c_encode) and k != 'volume_data': + if isinstance(xc, dict) or isinstance(xc, list): + c = self.get_learned_conditioning(xc) + else: + c = self.get_learned_conditioning(xc.to(self.device)) + else: + c = xc + if bs is not None: + c = c[:bs] + + if self.use_positional_encodings: + pos_x, pos_y = self.compute_latent_shifts(batch) + ckey = __conditioning_keys__[self.model.conditioning_key] + c = {ckey: c, 'pos_x': pos_x, 'pos_y': pos_y} + + else: + c = None + xc = None + if self.use_positional_encodings: + pos_x, pos_y = self.compute_latent_shifts(batch) + c = {'pos_x': pos_x, 'pos_y': pos_y} + out = [z, c] + if return_first_stage_outputs: + xrec = self.decode_first_stage(z) + out.extend([x, xrec]) + if return_x: + out.extend([x]) + if return_original_cond: + out.append(xc) + return out + + @torch.no_grad() + def decode_first_stage(self, z, predict_cids=False, force_not_quantize=False): + if predict_cids: + if z.dim() == 4: + z = torch.argmax(z.exp(), dim=1).long() + z = self.first_stage_model.quantize.get_codebook_entry(z, shape=None) + z = rearrange(z, 'b h w c -> b c h w').contiguous() + + z = 1. / self.scale_factor * z + return self.first_stage_model.decode(z) + + @torch.no_grad() + def encode_first_stage(self, x): + return self.first_stage_model.encode(x) + + def shared_step(self, batch, **kwargs): + x, c = self.get_input(batch, self.first_stage_key) + loss = self(x, c) + return loss + + def forward(self, x, c, *args, **kwargs): + if not self.fix_t: + t = torch.randint(0, self.num_timesteps, (x.shape[0],), device=self.device).long() + else: + t = torch.multinomial(weights, x.shape[0]//17, replacement=True) + t = t.reshape(((x.shape[0]//17,))).long().to(self.device) + t = t.unsqueeze(1) + t = t.repeat(1,17) + t = rearrange(t, 'b t-> (b t)') + # breakpoint() + if self.model.conditioning_key is not None: + assert c is not None + if self.cond_stage_trainable: + c = self.get_learned_conditioning(c) + if self.shorten_cond_schedule: # TODO: drop this option + tc = self.cond_ids[t].to(self.device) + c = self.q_sample(x_start=c, t=tc, noise=torch.randn_like(c.float())) + return self.p_losses(x, c, t, *args, **kwargs) + + def apply_model(self, x_noisy, t, cond, return_ids=False): + if isinstance(cond, dict): + # hybrid case, cond is expected to be a dict + pass + else: + if not isinstance(cond, list): + cond = [cond] + key = 'c_concat' if self.model.conditioning_key == 'concat' else 'c_crossattn' + cond = {key: cond} + + x_recon = self.model(x_noisy, t, **cond) + + if isinstance(x_recon, tuple) and not return_ids: + return x_recon[0] + else: + return x_recon + + def _predict_eps_from_xstart(self, x_t, t, pred_xstart): + return (extract_into_tensor(self.sqrt_recip_alphas_cumprod, t, x_t.shape) * x_t - pred_xstart) / \ + extract_into_tensor(self.sqrt_recipm1_alphas_cumprod, t, x_t.shape) + + def _prior_bpd(self, x_start): + """ + Get the prior KL term for the variational lower-bound, measured in + bits-per-dim. + This term can't be optimized, as it only depends on the encoder. + :param x_start: the [N x C x ...] tensor of inputs. + :return: a batch of [N] KL values (in bits), one per batch element. + """ + batch_size = x_start.shape[0] + t = torch.tensor([self.num_timesteps - 1] * batch_size, device=x_start.device) + qt_mean, _, qt_log_variance = self.q_mean_variance(x_start, t) + kl_prior = normal_kl(mean1=qt_mean, logvar1=qt_log_variance, mean2=0.0, logvar2=0.0) + return mean_flat(kl_prior) / np.log(2.0) + + def p_losses(self, x_start, cond, t, noise=None): + noise = default(noise, lambda: torch.randn_like(x_start)) + x_noisy = self.q_sample(x_start=x_start, t=t, noise=noise) + model_output = self.apply_model(x_noisy, t, cond) + + loss_dict = {} + prefix = 'train' if self.training else 'val' + + if self.parameterization == "x0": + target = x_start + elif self.parameterization == "eps": + target = noise + elif self.parameterization == "v": + target = self.get_v(x_start, noise, t) + else: + raise NotImplementedError() + + loss_simple = self.get_loss(model_output, target, mean=False).mean([1, 2, 3]) + loss_dict.update({f'{prefix}/loss_simple': loss_simple.mean()}) + + logvar_t = self.logvar[t].to(self.device) + loss = loss_simple / torch.exp(logvar_t) + logvar_t + # loss = loss_simple / torch.exp(self.logvar) + self.logvar + if self.learn_logvar: + loss_dict.update({f'{prefix}/loss_gamma': loss.mean()}) + loss_dict.update({'logvar': self.logvar.data.mean()}) + + loss = self.l_simple_weight * loss.mean() + + loss_vlb = self.get_loss(model_output, target, mean=False).mean(dim=(1, 2, 3)) + loss_vlb = (self.lvlb_weights[t] * loss_vlb).mean() + loss_dict.update({f'{prefix}/loss_vlb': loss_vlb}) + loss += (self.original_elbo_weight * loss_vlb) + loss_dict.update({f'{prefix}/loss': loss}) + + return loss, loss_dict + + def p_mean_variance(self, x, c, t, clip_denoised: bool, return_codebook_ids=False, quantize_denoised=False, + return_x0=False, score_corrector=None, corrector_kwargs=None): + t_in = t + model_out = self.apply_model(x, t_in, c, return_ids=return_codebook_ids) + + if score_corrector is not None: + assert self.parameterization == "eps" + model_out = score_corrector.modify_score(self, model_out, x, t, c, **corrector_kwargs) + + if return_codebook_ids: + model_out, logits = model_out + + if self.parameterization == "eps": + x_recon = self.predict_start_from_noise(x, t=t, noise=model_out) + elif self.parameterization == "x0": + x_recon = model_out + else: + raise NotImplementedError() + + if clip_denoised: + x_recon.clamp_(-1., 1.) + if quantize_denoised: + x_recon, _, [_, _, indices] = self.first_stage_model.quantize(x_recon) + model_mean, posterior_variance, posterior_log_variance = self.q_posterior(x_start=x_recon, x_t=x, t=t) + if return_codebook_ids: + return model_mean, posterior_variance, posterior_log_variance, logits + elif return_x0: + return model_mean, posterior_variance, posterior_log_variance, x_recon + else: + return model_mean, posterior_variance, posterior_log_variance + + @torch.no_grad() + def p_sample(self, x, c, t, clip_denoised=False, repeat_noise=False, + return_codebook_ids=False, quantize_denoised=False, return_x0=False, + temperature=1., noise_dropout=0., score_corrector=None, corrector_kwargs=None): + b, *_, device = *x.shape, x.device + outputs = self.p_mean_variance(x=x, c=c, t=t, clip_denoised=clip_denoised, + return_codebook_ids=return_codebook_ids, + quantize_denoised=quantize_denoised, + return_x0=return_x0, + score_corrector=score_corrector, corrector_kwargs=corrector_kwargs) + if return_codebook_ids: + raise DeprecationWarning("Support dropped.") + model_mean, _, model_log_variance, logits = outputs + elif return_x0: + model_mean, _, model_log_variance, x0 = outputs + else: + model_mean, _, model_log_variance = outputs + + noise = noise_like(x.shape, device, repeat_noise) * temperature + if noise_dropout > 0.: + noise = torch.nn.functional.dropout(noise, p=noise_dropout) + # no noise when t == 0 + nonzero_mask = (1 - (t == 0).float()).reshape(b, *((1,) * (len(x.shape) - 1))) + + if return_codebook_ids: + return model_mean + nonzero_mask * (0.5 * model_log_variance).exp() * noise, logits.argmax(dim=1) + if return_x0: + return model_mean + nonzero_mask * (0.5 * model_log_variance).exp() * noise, x0 + else: + return model_mean + nonzero_mask * (0.5 * model_log_variance).exp() * noise + + @torch.no_grad() + def progressive_denoising(self, cond, shape, verbose=True, callback=None, quantize_denoised=False, + img_callback=None, mask=None, x0=None, temperature=1., noise_dropout=0., + score_corrector=None, corrector_kwargs=None, batch_size=None, x_T=None, start_T=None, + log_every_t=None): + if not log_every_t: + log_every_t = self.log_every_t + timesteps = self.num_timesteps + if batch_size is not None: + b = batch_size if batch_size is not None else shape[0] + shape = [batch_size] + list(shape) + else: + b = batch_size = shape[0] + if x_T is None: + img = torch.randn(shape, device=self.device) + else: + img = x_T + intermediates = [] + if cond is not None: + if isinstance(cond, dict): + cond = {key: cond[key][:batch_size] if not isinstance(cond[key], list) else + list(map(lambda x: x[:batch_size], cond[key])) for key in cond} + else: + cond = [c[:batch_size] for c in cond] if isinstance(cond, list) else cond[:batch_size] + + if start_T is not None: + timesteps = min(timesteps, start_T) + iterator = tqdm(reversed(range(0, timesteps)), desc='Progressive Generation', + total=timesteps) if verbose else reversed( + range(0, timesteps)) + if type(temperature) == float: + temperature = [temperature] * timesteps + + for i in iterator: + ts = torch.full((b,), i, device=self.device, dtype=torch.long) + if self.shorten_cond_schedule: + assert self.model.conditioning_key != 'hybrid' + tc = self.cond_ids[ts].to(cond.device) + cond = self.q_sample(x_start=cond, t=tc, noise=torch.randn_like(cond)) + + img, x0_partial = self.p_sample(img, cond, ts, + clip_denoised=self.clip_denoised, + quantize_denoised=quantize_denoised, return_x0=True, + temperature=temperature[i], noise_dropout=noise_dropout, + score_corrector=score_corrector, corrector_kwargs=corrector_kwargs) + if mask is not None: + assert x0 is not None + img_orig = self.q_sample(x0, ts) + img = img_orig * mask + (1. - mask) * img + + if i % log_every_t == 0 or i == timesteps - 1: + intermediates.append(x0_partial) + if callback: callback(i) + if img_callback: img_callback(img, i) + return img, intermediates + + @torch.no_grad() + def p_sample_loop(self, cond, shape, return_intermediates=False, + x_T=None, verbose=True, callback=None, timesteps=None, quantize_denoised=False, + mask=None, x0=None, img_callback=None, start_T=None, + log_every_t=None): + + if not log_every_t: + log_every_t = self.log_every_t + device = self.betas.device + b = shape[0] + if x_T is None: + img = torch.randn(shape, device=device) + else: + img = x_T + + intermediates = [img] + if timesteps is None: + timesteps = self.num_timesteps + + if start_T is not None: + timesteps = min(timesteps, start_T) + iterator = tqdm(reversed(range(0, timesteps)), desc='Sampling t', total=timesteps) if verbose else reversed( + range(0, timesteps)) + + if mask is not None: + assert x0 is not None + assert x0.shape[2:3] == mask.shape[2:3] # spatial size has to match + + for i in iterator: + ts = torch.full((b,), i, device=device, dtype=torch.long) + if self.shorten_cond_schedule: + assert self.model.conditioning_key != 'hybrid' + tc = self.cond_ids[ts].to(cond.device) + cond = self.q_sample(x_start=cond, t=tc, noise=torch.randn_like(cond)) + + img = self.p_sample(img, cond, ts, + clip_denoised=self.clip_denoised, + quantize_denoised=quantize_denoised) + if mask is not None: + img_orig = self.q_sample(x0, ts) + img = img_orig * mask + (1. - mask) * img + + if i % log_every_t == 0 or i == timesteps - 1: + intermediates.append(img) + if callback: callback(i) + if img_callback: img_callback(img, i) + + if return_intermediates: + return img, intermediates + return img + + @torch.no_grad() + def sample(self, cond, batch_size=16, return_intermediates=False, x_T=None, + verbose=True, timesteps=None, quantize_denoised=False, + mask=None, x0=None, shape=None, **kwargs): + if shape is None: + shape = (batch_size, self.channels, self.image_size, self.image_size) + if cond is not None: + if isinstance(cond, dict): + cond = {key: cond[key][:batch_size] if not isinstance(cond[key], list) else + list(map(lambda x: x[:batch_size], cond[key])) for key in cond} + else: + cond = [c[:batch_size] for c in cond] if isinstance(cond, list) else cond[:batch_size] + return self.p_sample_loop(cond, + shape, + return_intermediates=return_intermediates, x_T=x_T, + verbose=verbose, timesteps=timesteps, quantize_denoised=quantize_denoised, + mask=mask, x0=x0) + + @torch.no_grad() + def sample_log(self, cond, batch_size, ddim, ddim_steps, **kwargs): + if ddim: + ddim_sampler = DDIMSampler(self) + if self.model.conditioning_key == 'crossattn' or self.model.conditioning_key == 'hybrid': + shape = (self.channels, self.image_size, self.image_size) + else: + shape = ((self.channels)//2, self.image_size, self.image_size) + samples, intermediates = ddim_sampler.sample(ddim_steps, batch_size, + shape, cond, verbose=False, **kwargs) + + else: + samples, intermediates = self.sample(cond=cond, batch_size=batch_size, + return_intermediates=True, **kwargs) + + return samples, intermediates + + @torch.no_grad() + def get_unconditional_conditioning(self, batch_size, null_label=None): + if null_label is not None: + xc = null_label + if isinstance(xc, ListConfig): + xc = list(xc) + if isinstance(xc, dict) or isinstance(xc, list): + c = self.get_learned_conditioning(xc) + else: + if hasattr(xc, "to"): + xc = xc.to(self.device) + c = self.get_learned_conditioning(xc) + else: + if self.cond_stage_key in ["class_label", "cls"]: + xc = self.cond_stage_model.get_unconditional_conditioning(batch_size, device=self.device) + return self.get_learned_conditioning(xc) + else: + raise NotImplementedError("todo") + if isinstance(c, list): # in case the encoder gives us a list + for i in range(len(c)): + c[i] = repeat(c[i], '1 ... -> b ...', b=batch_size).to(self.device) + else: + c = repeat(c, '1 ... -> b ...', b=batch_size).to(self.device) + return c + + @torch.no_grad() + def log_images(self, batch, N=16, n_row=4, sample=True, ddim_steps=50, ddim_eta=0., return_keys=None, + quantize_denoised=True, inpaint=True, plot_denoise_rows=False, plot_progressive_rows=True, + plot_diffusion_rows=True, unconditional_guidance_scale=1., unconditional_guidance_label=None, + use_ema_scope=True, + **kwargs): + ema_scope = self.ema_scope if use_ema_scope else nullcontext + use_ddim = ddim_steps is not None + + if self.fix_t == True: + sample = True + plot_denoise_rows = False + plot_progressive_rows = False + plot_diffusion_rows = False + inpaint = False + + log = dict() + z, c, x, xrec, xc = self.get_input(batch, self.first_stage_key, + return_first_stage_outputs=True, + force_c_encode=True, + return_original_cond=True, + bs=N) + N = min(x.shape[0], N) + n_row = min(x.shape[0], n_row) + log["inputs"] = x + log["reconstruction"] = xrec + # if self.model.conditioning_key is not None: + # if hasattr(self.cond_stage_model, "decode"): + # # xc = self.cond_stage_model.decode(c) + # if c.shape[1] == 8: + # xc = self.decode_first_stage(c[:,:4]) + # elif c.shape[1] == 6: + # xc = self.decode_first_stage(c[:, :3]) + # else: + # xc = self.decode_first_stage(c) + # log["conditioning"] = xc + # elif self.cond_stage_key in ["caption", "txt"]: + # xc = log_txt_as_img((x.shape[2], x.shape[3]), batch[self.cond_stage_key], size=x.shape[2] // 25) + # log["conditioning"] = xc + # elif self.cond_stage_key in ['class_label', "cls"]: + # try: + # xc = log_txt_as_img((x.shape[2], x.shape[3]), batch["human_label"], size=x.shape[2] // 25) + # log['conditioning'] = xc + # except KeyError: + # # probably no "human_label" in batch + # pass + # elif isimage(xc): + # log["conditioning"] = xc + # if ismap(xc): + # log["original_conditioning"] = self.to_rgb(xc) + + if plot_diffusion_rows: + # get diffusion row + diffusion_row = list() + z_start = z[:n_row] + for t in range(self.num_timesteps): + if t % self.log_every_t == 0 or t == self.num_timesteps - 1: + t = repeat(torch.tensor([t]), '1 -> b', b=n_row) + t = t.to(self.device).long() + noise = torch.randn_like(z_start) + z_noisy = self.q_sample(x_start=z_start, t=t, noise=noise) + diffusion_row.append(self.decode_first_stage(z_noisy)) + + diffusion_row = torch.stack(diffusion_row) # n_log_step, n_row, C, H, W + diffusion_grid = rearrange(diffusion_row, 'n b c h w -> b n c h w') + diffusion_grid = rearrange(diffusion_grid, 'b n c h w -> (b n) c h w') + diffusion_grid = make_grid(diffusion_grid, nrow=diffusion_row.shape[0]) + log["diffusion_row"] = diffusion_grid + + if sample: + # get denoise row + with ema_scope("Sampling"): + samples, z_denoise_row = self.sample_log(cond=c, batch_size=N, ddim=use_ddim, + ddim_steps=ddim_steps, eta=ddim_eta) + # samples, z_denoise_row = self.sample(cond=c, batch_size=N, return_intermediates=True) + x_samples = self.decode_first_stage(samples) + log["samples"] = x_samples + if plot_denoise_rows: + denoise_grid = self._get_denoise_row_from_list(z_denoise_row) + log["denoise_row"] = denoise_grid + + if quantize_denoised and not isinstance(self.first_stage_model, AutoencoderKL) and not isinstance( + self.first_stage_model, IdentityFirstStage): + # also display when quantizing x0 while sampling + with ema_scope("Plotting Quantized Denoised"): + samples, z_denoise_row = self.sample_log(cond=c, batch_size=N, ddim=use_ddim, + ddim_steps=ddim_steps, eta=ddim_eta, + quantize_denoised=True) + # samples, z_denoise_row = self.sample(cond=c, batch_size=N, return_intermediates=True, + # quantize_denoised=True) + x_samples = self.decode_first_stage(samples.to(self.device)) + log["samples_x0_quantized"] = x_samples + + if unconditional_guidance_scale > 1.0: + uc = self.get_unconditional_conditioning(N, unconditional_guidance_label) + if self.model.conditioning_key == "crossattn-adm": + uc = {"c_crossattn": [uc], "c_adm": c["c_adm"]} + with ema_scope("Sampling with classifier-free guidance"): + samples_cfg, _ = self.sample_log(cond=c, batch_size=N, ddim=use_ddim, + ddim_steps=ddim_steps, eta=ddim_eta, + unconditional_guidance_scale=unconditional_guidance_scale, + unconditional_conditioning=uc, + ) + x_samples_cfg = self.decode_first_stage(samples_cfg) + log[f"samples_cfg_scale_{unconditional_guidance_scale:.2f}"] = x_samples_cfg + + if c.shape[1] == 8 or c.shape[1] == 6: + inpaint = False + if inpaint: + # make a simple center square + b, h, w = z.shape[0], z.shape[2], z.shape[3] + mask = torch.ones(N, h, w).to(self.device) + # zeros will be filled in + mask[:, h // 4:3 * h // 4, w // 4:3 * w // 4] = 0. + mask = mask[:, None, ...] + with ema_scope("Plotting Inpaint"): + samples, _ = self.sample_log(cond=c, batch_size=N, ddim=use_ddim, eta=ddim_eta, + ddim_steps=ddim_steps, x0=z[:N], mask=mask) + x_samples = self.decode_first_stage(samples.to(self.device)) + log["samples_inpainting"] = x_samples + log["mask"] = mask + + # outpaint + mask = 1. - mask + with ema_scope("Plotting Outpaint"): + samples, _ = self.sample_log(cond=c, batch_size=N, ddim=use_ddim, eta=ddim_eta, + ddim_steps=ddim_steps, x0=z[:N], mask=mask) + x_samples = self.decode_first_stage(samples.to(self.device)) + log["samples_outpainting"] = x_samples + + if plot_progressive_rows: + with ema_scope("Plotting Progressives"): + img, progressives = self.progressive_denoising(c, + shape=(self.channels, self.image_size, self.image_size), + batch_size=N) + prog_row = self._get_denoise_row_from_list(progressives, desc="Progressive Generation") + log["progressive_row"] = prog_row + + if return_keys: + if np.intersect1d(list(log.keys()), return_keys).shape[0] == 0: + return log + else: + return {key: log[key] for key in return_keys} + return log + + def configure_optimizers(self): + lr = self.learning_rate + params = [] + for name, p in self.model.named_parameters(): + if 'tem' in name: + params.append(p) + # params = list(self.model.parameters()) + if self.cond_stage_trainable: + print(f"{self.__class__.__name__}: Also optimizing conditioner params!") + params = params + list(self.cond_stage_model.parameters()) + if self.learn_logvar: + print('Diffusion model optimizing logvar') + params.append(self.logvar) + opt = torch.optim.AdamW(params, lr=lr) + if self.use_scheduler: + assert 'target' in self.scheduler_config + scheduler = instantiate_from_config(self.scheduler_config) + + print("Setting up LambdaLR scheduler...") + scheduler = [ + { + 'scheduler': LambdaLR(opt, lr_lambda=scheduler.schedule), + 'interval': 'step', + 'frequency': 1 + }] + return [opt], scheduler + return opt + + @torch.no_grad() + def to_rgb(self, x): + x = x.float() + if not hasattr(self, "colorize"): + self.colorize = torch.randn(3, x.shape[1], 1, 1).to(x) + x = nn.functional.conv2d(x, weight=self.colorize) + x = 2. * (x - x.min()) / (x.max() - x.min()) - 1. + return x + + +class DiffusionWrapper(pl.LightningModule): + def __init__(self, diff_model_config, conditioning_key): + super().__init__() + self.sequential_cross_attn = diff_model_config.pop("sequential_crossattn", False) + self.diffusion_model = instantiate_from_config(diff_model_config) + self.conditioning_key = conditioning_key + assert self.conditioning_key in [None, 'concat', 'crossattn', 'hybrid', 'adm', 'hybrid-adm', 'crossattn-adm'] + + def forward(self, x, t, c_concat: list = None, c_crossattn: list = None, c_adm=None): + if self.conditioning_key is None: + out = self.diffusion_model(x, t) + elif self.conditioning_key == 'concat': + # breakpoint() + xc = torch.cat([x] + c_concat, dim=1) + out = self.diffusion_model(xc, t) + elif self.conditioning_key == 'crossattn': + # if not self.sequential_cross_attn: + # cc = torch.cat(c_crossattn, 1) + # else: + # cc = c_crossattn + # out = self.diffusion_model(x, t, context=cc) + # breakpoint() + xc = torch.cat([x] + [c_concat], dim=1) + cc = torch.cat([c_crossattn], 1) + out = self.diffusion_model(xc, t, context=cc) + elif self.conditioning_key == 'hybrid': + xc = torch.cat([x] + c_crossattn, dim=1) + cc = torch.cat(c_crossattn, 1) + out = self.diffusion_model(xc, t, context=cc) + elif self.conditioning_key == 'hybrid-adm': + assert c_adm is not None + xc = torch.cat([x] + c_concat, dim=1) + cc = torch.cat(c_crossattn, 1) + out = self.diffusion_model(xc, t, context=cc, y=c_adm) + elif self.conditioning_key == 'crossattn-adm': + assert c_adm is not None + cc = torch.cat(c_crossattn, 1) + out = self.diffusion_model(x, t, context=cc, y=c_adm) + elif self.conditioning_key == 'adm': + cc = c_crossattn[0] + out = self.diffusion_model(x, t, y=cc) + else: + raise NotImplementedError() + + return out diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm_recon.py b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm_recon.py new file mode 100644 index 0000000000000000000000000000000000000000..af03934332de891dee007853077d9de5f57d1641 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm_recon.py @@ -0,0 +1,141 @@ +""" +wild mixture of +https://github.com/lucidrains/denoising-diffusion-pytorch/blob/7706bdfc6f527f58d33f84b7b522e61e6e3164b3/denoising_diffusion_pytorch/denoising_diffusion_pytorch.py +https://github.com/openai/improved-diffusion/blob/e94489283bb876ac1477d5dd7709bbbd2d9902ce/improved_diffusion/gaussian_diffusion.py +https://github.com/CompVis/taming-transformers +-- merci +""" + +import torch +import torch.nn as nn +import numpy as np +import pytorch_lightning as pl +from torch.optim.lr_scheduler import LambdaLR +from einops import rearrange, repeat +from contextlib import contextmanager, nullcontext +from functools import partial +import itertools +from tqdm import tqdm +from torchvision.utils import make_grid +from pytorch_lightning.utilities.distributed import rank_zero_only +from omegaconf import ListConfig + +from ldm.util import log_txt_as_img, exists, default, ismap, isimage, mean_flat, count_params, instantiate_from_config +from ldm.modules.ema import LitEma +from ldm.modules.distributions.distributions import normal_kl, DiagonalGaussianDistribution +from ldm.models.autoencoder import IdentityFirstStage, AutoencoderKL +from ldm.modules.diffusionmodules.util import make_beta_schedule, extract_into_tensor, noise_like +from ldm.models.diffusion.ddim import DDIMSampler + + +__conditioning_keys__ = {'concat': 'c_concat', + 'crossattn': 'c_crossattn', + 'adm': 'y'} + + +def disabled_train(self, mode=True): + """Overwrite model.train with this function to make sure train/eval mode + does not change anymore.""" + return self + + +def uniform_on_device(r1, r2, shape, device): + return (r1 - r2) * torch.rand(*shape, device=device) + r2 + + + +class Latent_Recon(pl.LightningModule): + """main class""" + + def __init__(self, + first_stage_config, + cond_stage_config, + num_timesteps_cond=None, + cond_stage_key="image", + cond_stage_trainable=False, + concat_mode=True, + cond_stage_forward=None, + conditioning_key=None, + scale_factor=1.0, + scale_by_std=False, + force_null_conditioning=False, + *args, **kwargs): + super().__init__() + self.scale_by_std = scale_by_std + + try: + self.num_downs = len(first_stage_config.params.ddconfig.ch_mult) - 1 + except: + self.num_downs = 0 + if not scale_by_std: + self.scale_factor = scale_factor + else: + self.register_buffer('scale_factor', torch.tensor(scale_factor)) + # breakpoint() + self.instantiate_first_stage(first_stage_config) + + def instantiate_first_stage(self, config): + model = instantiate_from_config(config) + self.first_stage_model = model.eval() + self.first_stage_model.train = disabled_train + for param in self.first_stage_model.parameters(): + param.requires_grad = False + + def get_first_stage_encoding(self, encoder_posterior): + # breakpoint() + if isinstance(encoder_posterior, DiagonalGaussianDistribution): + z = encoder_posterior.sample() + elif isinstance(encoder_posterior, torch.Tensor): + z = encoder_posterior + else: + raise NotImplementedError(f"encoder_posterior of type '{type(encoder_posterior)}' not yet implemented") + return self.scale_factor * z + + + @torch.no_grad() + def get_input(self, batch): + + x = rearrange(batch, 'b h w c -> b c h w') + x = x.repeat(1,3,1,1) + x = x.to(memory_format=torch.contiguous_format).float() + + x = x.to(self.device) + encoder_posterior = self.encode_first_stage(x) + z = self.get_first_stage_encoding(encoder_posterior).detach() + + xrec = self.decode_first_stage(z) + return xrec + + @torch.no_grad() + def decode_first_stage(self, z, predict_cids=False, force_not_quantize=False): + if predict_cids: + if z.dim() == 4: + z = torch.argmax(z.exp(), dim=1).long() + z = self.first_stage_model.quantize.get_codebook_entry(z, shape=None) + z = rearrange(z, 'b h w c -> b c h w').contiguous() + + z = 1. / self.scale_factor * z + return self.first_stage_model.decode(z) + + @torch.no_grad() + def encode_first_stage(self, x): + return self.first_stage_model.encode(x) + + def forward(self, batch, *args, **kwargs): + xrec = self.get_input(batch) + return xrec + + @torch.no_grad() + def validation_step(self, batch, batch_idx): + xrec = self.forward(batch) + return xrec + + @torch.no_grad() + def to_rgb(self, x): + x = x.float() + if not hasattr(self, "colorize"): + self.colorize = torch.randn(3, x.shape[1], 1, 1).to(x) + x = nn.functional.conv2d(x, weight=self.colorize) + x = 2. * (x - x.min()) / (x.max() - x.min()) - 1. + return x + diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm_recon_vq.py b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm_recon_vq.py new file mode 100644 index 0000000000000000000000000000000000000000..7bf3d28fb83fafcf8a2c3017df6c717f9eacc1f9 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/ddpm_recon_vq.py @@ -0,0 +1,144 @@ +""" +wild mixture of +https://github.com/lucidrains/denoising-diffusion-pytorch/blob/7706bdfc6f527f58d33f84b7b522e61e6e3164b3/denoising_diffusion_pytorch/denoising_diffusion_pytorch.py +https://github.com/openai/improved-diffusion/blob/e94489283bb876ac1477d5dd7709bbbd2d9902ce/improved_diffusion/gaussian_diffusion.py +https://github.com/CompVis/taming-transformers +-- merci +""" + +import torch +import torch.nn as nn +import numpy as np +import pytorch_lightning as pl +from torch.optim.lr_scheduler import LambdaLR +from einops import rearrange, repeat +from contextlib import contextmanager, nullcontext +from functools import partial +import itertools +from tqdm import tqdm +from torchvision.utils import make_grid +from pytorch_lightning.utilities.distributed import rank_zero_only +from omegaconf import ListConfig + +from ldm.util import log_txt_as_img, exists, default, ismap, isimage, mean_flat, count_params, instantiate_from_config +from ldm.modules.ema import LitEma +from ldm.modules.distributions.distributions import normal_kl, DiagonalGaussianDistribution +from ldm.models.autoencoder import IdentityFirstStage, AutoencoderKL +from ldm.modules.diffusionmodules.util import make_beta_schedule, extract_into_tensor, noise_like +from ldm.models.diffusion.ddim import DDIMSampler + + +__conditioning_keys__ = {'concat': 'c_concat', + 'crossattn': 'c_crossattn', + 'adm': 'y'} + + +def disabled_train(self, mode=True): + """Overwrite model.train with this function to make sure train/eval mode + does not change anymore.""" + return self + + +def uniform_on_device(r1, r2, shape, device): + return (r1 - r2) * torch.rand(*shape, device=device) + r2 + + + +class Latent_Recon(pl.LightningModule): + """main class""" + + def __init__(self, + first_stage_config, + cond_stage_config, + num_timesteps_cond=None, + cond_stage_key="image", + cond_stage_trainable=False, + concat_mode=True, + cond_stage_forward=None, + conditioning_key=None, + scale_factor=1.0, + scale_by_std=False, + force_null_conditioning=False, + *args, **kwargs): + super().__init__() + self.scale_by_std = scale_by_std + + try: + self.num_downs = len(first_stage_config.params.ddconfig.ch_mult) - 1 + except: + self.num_downs = 0 + if not scale_by_std: + self.scale_factor = scale_factor + else: + self.register_buffer('scale_factor', torch.tensor(scale_factor)) + # breakpoint() + self.instantiate_first_stage(first_stage_config) + + def instantiate_first_stage(self, config): + model = instantiate_from_config(config) + self.first_stage_model = model.eval() + self.first_stage_model.train = disabled_train + for param in self.first_stage_model.parameters(): + param.requires_grad = False + + def get_first_stage_encoding(self, encoder_posterior): + # breakpoint() + if isinstance(encoder_posterior, DiagonalGaussianDistribution): + z = encoder_posterior.sample() + elif isinstance(encoder_posterior, torch.Tensor): + z = encoder_posterior + elif isinstance(encoder_posterior, tuple): + z = encoder_posterior[0] + else: + raise NotImplementedError(f"encoder_posterior of type '{type(encoder_posterior)}' not yet implemented") + # breakpoint() + return self.scale_factor * z + + + @torch.no_grad() + def get_input(self, batch): + + x = rearrange(batch, 'b h w c -> b c h w') + x = x.repeat(1,3,1,1) + x = x.to(memory_format=torch.contiguous_format).float() + + x = x.to(self.device) + encoder_posterior = self.encode_first_stage(x) + z = self.get_first_stage_encoding(encoder_posterior).detach() + + xrec = self.decode_first_stage(z) + return xrec + + @torch.no_grad() + def decode_first_stage(self, z, predict_cids=False, force_not_quantize=False): + if predict_cids: + if z.dim() == 4: + z = torch.argmax(z.exp(), dim=1).long() + z = self.first_stage_model.quantize.get_codebook_entry(z, shape=None) + z = rearrange(z, 'b h w c -> b c h w').contiguous() + + z = 1. / self.scale_factor * z + return self.first_stage_model.decode(z) + + @torch.no_grad() + def encode_first_stage(self, x): + return self.first_stage_model.encode(x) + + def forward(self, batch, *args, **kwargs): + xrec = self.get_input(batch) + return xrec + + @torch.no_grad() + def validation_step(self, batch, batch_idx): + xrec = self.forward(batch) + return xrec + + @torch.no_grad() + def to_rgb(self, x): + x = x.float() + if not hasattr(self, "colorize"): + self.colorize = torch.randn(3, x.shape[1], 1, 1).to(x) + x = nn.functional.conv2d(x, weight=self.colorize) + x = 2. * (x - x.min()) / (x.max() - x.min()) - 1. + return x + diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/plms.py b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/plms.py new file mode 100644 index 0000000000000000000000000000000000000000..7002a365d27168ced0a04e9a4d83e088f8284eae --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/plms.py @@ -0,0 +1,244 @@ +"""SAMPLING ONLY.""" + +import torch +import numpy as np +from tqdm import tqdm +from functools import partial + +from ldm.modules.diffusionmodules.util import make_ddim_sampling_parameters, make_ddim_timesteps, noise_like +from ldm.models.diffusion.sampling_util import norm_thresholding + + +class PLMSSampler(object): + def __init__(self, model, schedule="linear", **kwargs): + super().__init__() + self.model = model + self.ddpm_num_timesteps = model.num_timesteps + self.schedule = schedule + + def register_buffer(self, name, attr): + if type(attr) == torch.Tensor: + if attr.device != torch.device("cuda"): + attr = attr.to(torch.device("cuda")) + setattr(self, name, attr) + + def make_schedule(self, ddim_num_steps, ddim_discretize="uniform", ddim_eta=0., verbose=True): + if ddim_eta != 0: + raise ValueError('ddim_eta must be 0 for PLMS') + self.ddim_timesteps = make_ddim_timesteps(ddim_discr_method=ddim_discretize, num_ddim_timesteps=ddim_num_steps, + num_ddpm_timesteps=self.ddpm_num_timesteps,verbose=verbose) + alphas_cumprod = self.model.alphas_cumprod + assert alphas_cumprod.shape[0] == self.ddpm_num_timesteps, 'alphas have to be defined for each timestep' + to_torch = lambda x: x.clone().detach().to(torch.float32).to(self.model.device) + + self.register_buffer('betas', to_torch(self.model.betas)) + self.register_buffer('alphas_cumprod', to_torch(alphas_cumprod)) + self.register_buffer('alphas_cumprod_prev', to_torch(self.model.alphas_cumprod_prev)) + + # calculations for diffusion q(x_t | x_{t-1}) and others + self.register_buffer('sqrt_alphas_cumprod', to_torch(np.sqrt(alphas_cumprod.cpu()))) + self.register_buffer('sqrt_one_minus_alphas_cumprod', to_torch(np.sqrt(1. - alphas_cumprod.cpu()))) + self.register_buffer('log_one_minus_alphas_cumprod', to_torch(np.log(1. - alphas_cumprod.cpu()))) + self.register_buffer('sqrt_recip_alphas_cumprod', to_torch(np.sqrt(1. / alphas_cumprod.cpu()))) + self.register_buffer('sqrt_recipm1_alphas_cumprod', to_torch(np.sqrt(1. / alphas_cumprod.cpu() - 1))) + + # ddim sampling parameters + ddim_sigmas, ddim_alphas, ddim_alphas_prev = make_ddim_sampling_parameters(alphacums=alphas_cumprod.cpu(), + ddim_timesteps=self.ddim_timesteps, + eta=ddim_eta,verbose=verbose) + self.register_buffer('ddim_sigmas', ddim_sigmas) + self.register_buffer('ddim_alphas', ddim_alphas) + self.register_buffer('ddim_alphas_prev', ddim_alphas_prev) + self.register_buffer('ddim_sqrt_one_minus_alphas', np.sqrt(1. - ddim_alphas)) + sigmas_for_original_sampling_steps = ddim_eta * torch.sqrt( + (1 - self.alphas_cumprod_prev) / (1 - self.alphas_cumprod) * ( + 1 - self.alphas_cumprod / self.alphas_cumprod_prev)) + self.register_buffer('ddim_sigmas_for_original_num_steps', sigmas_for_original_sampling_steps) + + @torch.no_grad() + def sample(self, + S, + batch_size, + shape, + conditioning=None, + callback=None, + normals_sequence=None, + img_callback=None, + quantize_x0=False, + eta=0., + mask=None, + x0=None, + temperature=1., + noise_dropout=0., + score_corrector=None, + corrector_kwargs=None, + verbose=True, + x_T=None, + log_every_t=100, + unconditional_guidance_scale=1., + unconditional_conditioning=None, + # this has to come in the same format as the conditioning, # e.g. as encoded tokens, ... + dynamic_threshold=None, + **kwargs + ): + if conditioning is not None: + if isinstance(conditioning, dict): + cbs = conditioning[list(conditioning.keys())[0]].shape[0] + if cbs != batch_size: + print(f"Warning: Got {cbs} conditionings but batch-size is {batch_size}") + else: + if conditioning.shape[0] != batch_size: + print(f"Warning: Got {conditioning.shape[0]} conditionings but batch-size is {batch_size}") + + self.make_schedule(ddim_num_steps=S, ddim_eta=eta, verbose=verbose) + # sampling + C, H, W = shape + size = (batch_size, C, H, W) + print(f'Data shape for PLMS sampling is {size}') + + samples, intermediates = self.plms_sampling(conditioning, size, + callback=callback, + img_callback=img_callback, + quantize_denoised=quantize_x0, + mask=mask, x0=x0, + ddim_use_original_steps=False, + noise_dropout=noise_dropout, + temperature=temperature, + score_corrector=score_corrector, + corrector_kwargs=corrector_kwargs, + x_T=x_T, + log_every_t=log_every_t, + unconditional_guidance_scale=unconditional_guidance_scale, + unconditional_conditioning=unconditional_conditioning, + dynamic_threshold=dynamic_threshold, + ) + return samples, intermediates + + @torch.no_grad() + def plms_sampling(self, cond, shape, + x_T=None, ddim_use_original_steps=False, + callback=None, timesteps=None, quantize_denoised=False, + mask=None, x0=None, img_callback=None, log_every_t=100, + temperature=1., noise_dropout=0., score_corrector=None, corrector_kwargs=None, + unconditional_guidance_scale=1., unconditional_conditioning=None, + dynamic_threshold=None): + device = self.model.betas.device + b = shape[0] + if x_T is None: + img = torch.randn(shape, device=device) + else: + img = x_T + + if timesteps is None: + timesteps = self.ddpm_num_timesteps if ddim_use_original_steps else self.ddim_timesteps + elif timesteps is not None and not ddim_use_original_steps: + subset_end = int(min(timesteps / self.ddim_timesteps.shape[0], 1) * self.ddim_timesteps.shape[0]) - 1 + timesteps = self.ddim_timesteps[:subset_end] + + intermediates = {'x_inter': [img], 'pred_x0': [img]} + time_range = list(reversed(range(0,timesteps))) if ddim_use_original_steps else np.flip(timesteps) + total_steps = timesteps if ddim_use_original_steps else timesteps.shape[0] + print(f"Running PLMS Sampling with {total_steps} timesteps") + + iterator = tqdm(time_range, desc='PLMS Sampler', total=total_steps) + old_eps = [] + + for i, step in enumerate(iterator): + index = total_steps - i - 1 + ts = torch.full((b,), step, device=device, dtype=torch.long) + ts_next = torch.full((b,), time_range[min(i + 1, len(time_range) - 1)], device=device, dtype=torch.long) + + if mask is not None: + assert x0 is not None + img_orig = self.model.q_sample(x0, ts) # TODO: deterministic forward pass? + img = img_orig * mask + (1. - mask) * img + + outs = self.p_sample_plms(img, cond, ts, index=index, use_original_steps=ddim_use_original_steps, + quantize_denoised=quantize_denoised, temperature=temperature, + noise_dropout=noise_dropout, score_corrector=score_corrector, + corrector_kwargs=corrector_kwargs, + unconditional_guidance_scale=unconditional_guidance_scale, + unconditional_conditioning=unconditional_conditioning, + old_eps=old_eps, t_next=ts_next, + dynamic_threshold=dynamic_threshold) + img, pred_x0, e_t = outs + old_eps.append(e_t) + if len(old_eps) >= 4: + old_eps.pop(0) + if callback: callback(i) + if img_callback: img_callback(pred_x0, i) + + if index % log_every_t == 0 or index == total_steps - 1: + intermediates['x_inter'].append(img) + intermediates['pred_x0'].append(pred_x0) + + return img, intermediates + + @torch.no_grad() + def p_sample_plms(self, x, c, t, index, repeat_noise=False, use_original_steps=False, quantize_denoised=False, + temperature=1., noise_dropout=0., score_corrector=None, corrector_kwargs=None, + unconditional_guidance_scale=1., unconditional_conditioning=None, old_eps=None, t_next=None, + dynamic_threshold=None): + b, *_, device = *x.shape, x.device + + def get_model_output(x, t): + if unconditional_conditioning is None or unconditional_guidance_scale == 1.: + e_t = self.model.apply_model(x, t, c) + else: + x_in = torch.cat([x] * 2) + t_in = torch.cat([t] * 2) + c_in = torch.cat([unconditional_conditioning, c]) + e_t_uncond, e_t = self.model.apply_model(x_in, t_in, c_in).chunk(2) + e_t = e_t_uncond + unconditional_guidance_scale * (e_t - e_t_uncond) + + if score_corrector is not None: + assert self.model.parameterization == "eps" + e_t = score_corrector.modify_score(self.model, e_t, x, t, c, **corrector_kwargs) + + return e_t + + alphas = self.model.alphas_cumprod if use_original_steps else self.ddim_alphas + alphas_prev = self.model.alphas_cumprod_prev if use_original_steps else self.ddim_alphas_prev + sqrt_one_minus_alphas = self.model.sqrt_one_minus_alphas_cumprod if use_original_steps else self.ddim_sqrt_one_minus_alphas + sigmas = self.model.ddim_sigmas_for_original_num_steps if use_original_steps else self.ddim_sigmas + + def get_x_prev_and_pred_x0(e_t, index): + # select parameters corresponding to the currently considered timestep + a_t = torch.full((b, 1, 1, 1), alphas[index], device=device) + a_prev = torch.full((b, 1, 1, 1), alphas_prev[index], device=device) + sigma_t = torch.full((b, 1, 1, 1), sigmas[index], device=device) + sqrt_one_minus_at = torch.full((b, 1, 1, 1), sqrt_one_minus_alphas[index],device=device) + + # current prediction for x_0 + pred_x0 = (x - sqrt_one_minus_at * e_t) / a_t.sqrt() + if quantize_denoised: + pred_x0, _, *_ = self.model.first_stage_model.quantize(pred_x0) + if dynamic_threshold is not None: + pred_x0 = norm_thresholding(pred_x0, dynamic_threshold) + # direction pointing to x_t + dir_xt = (1. - a_prev - sigma_t**2).sqrt() * e_t + noise = sigma_t * noise_like(x.shape, device, repeat_noise) * temperature + if noise_dropout > 0.: + noise = torch.nn.functional.dropout(noise, p=noise_dropout) + x_prev = a_prev.sqrt() * pred_x0 + dir_xt + noise + return x_prev, pred_x0 + + e_t = get_model_output(x, t) + if len(old_eps) == 0: + # Pseudo Improved Euler (2nd order) + x_prev, pred_x0 = get_x_prev_and_pred_x0(e_t, index) + e_t_next = get_model_output(x_prev, t_next) + e_t_prime = (e_t + e_t_next) / 2 + elif len(old_eps) == 1: + # 2nd order Pseudo Linear Multistep (Adams-Bashforth) + e_t_prime = (3 * e_t - old_eps[-1]) / 2 + elif len(old_eps) == 2: + # 3nd order Pseudo Linear Multistep (Adams-Bashforth) + e_t_prime = (23 * e_t - 16 * old_eps[-1] + 5 * old_eps[-2]) / 12 + elif len(old_eps) >= 3: + # 4nd order Pseudo Linear Multistep (Adams-Bashforth) + e_t_prime = (55 * e_t - 59 * old_eps[-1] + 37 * old_eps[-2] - 9 * old_eps[-3]) / 24 + + x_prev, pred_x0 = get_x_prev_and_pred_x0(e_t_prime, index) + + return x_prev, pred_x0, e_t diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/sampling_util.py b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/sampling_util.py new file mode 100644 index 0000000000000000000000000000000000000000..7eff02be6d7c54d43ee6680636ac0698dd3b3f33 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/models/diffusion/sampling_util.py @@ -0,0 +1,22 @@ +import torch +import numpy as np + + +def append_dims(x, target_dims): + """Appends dimensions to the end of a tensor until it has target_dims dimensions. + From https://github.com/crowsonkb/k-diffusion/blob/master/k_diffusion/utils.py""" + dims_to_append = target_dims - x.ndim + if dims_to_append < 0: + raise ValueError(f'input has {x.ndim} dims but target_dims is {target_dims}, which is less') + return x[(...,) + (None,) * dims_to_append] + + +def norm_thresholding(x0, value): + s = append_dims(x0.pow(2).flatten(1).mean(1).sqrt().clamp(min=value), x0.ndim) + return x0 * (value / s) + + +def spatial_norm_thresholding(x0, value): + # b c h w + s = x0.pow(2).mean(1, keepdim=True).sqrt().clamp(min=value) + return x0 * (value / s) \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..81dd6424d8fb911c8e27c9720db7f8f8b15268e5 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..cc639916e15f3ea133dd6a701565804f581b4aeb Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..22c3e3bcb8395144e8c7bd913c3fccb1bc18a717 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..22d25e0d02b43d882ebab2521551425189c2b9da Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/attention.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..61dc03d83dcb56a956843d9138ca370ffa145813 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..a98c49ef69e5286f776e87c70cb43141eb7984aa Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..cb5d95d9d420f8dd12da9e371f2a4759ba154c6c Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..d29b8d7e063eba498a8eeaf700de1eb3e39b8a3d Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/ema.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/x_transformer.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/x_transformer.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..c2c2a2ec545e54abefcdaf23888e8f71b8d6d818 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/x_transformer.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/x_transformer.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/x_transformer.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..bf29a02ee530dabdb5a83ca9d0f4399dbba31b03 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/x_transformer.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/x_transformer.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/x_transformer.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..cc6d13c74e812e9ef09f0ba7b230505d15e0ce55 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/__pycache__/x_transformer.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/attention.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/attention.py new file mode 100644 index 0000000000000000000000000000000000000000..c7e06ed79f47b0b7b869531e69e1f9de6f69c2a7 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/modules/attention.py @@ -0,0 +1,354 @@ +from inspect import isfunction +import math +import torch +import torch.nn.functional as F +from torch import nn, einsum +from einops import rearrange, repeat +from typing import Optional, Any + +from ldm.modules.diffusionmodules.util import checkpoint + + +try: + import xformers + import xformers.ops + XFORMERS_IS_AVAILBLE = True +except: + XFORMERS_IS_AVAILBLE = False + +# CrossAttn precision handling +import os +_ATTN_PRECISION = os.environ.get("ATTN_PRECISION", "fp32") + +def exists(val): + return val is not None + + +def uniq(arr): + return{el: True for el in arr}.keys() + + +def default(val, d): + if exists(val): + return val + return d() if isfunction(d) else d + + +def max_neg_value(t): + return -torch.finfo(t.dtype).max + + +def init_(tensor): + dim = tensor.shape[-1] + std = 1 / math.sqrt(dim) + tensor.uniform_(-std, std) + return tensor + + +# feedforward +class GEGLU(nn.Module): + def __init__(self, dim_in, dim_out): + super().__init__() + self.proj = nn.Linear(dim_in, dim_out * 2) + + def forward(self, x): + x, gate = self.proj(x).chunk(2, dim=-1) + return x * F.gelu(gate) + + +class FeedForward(nn.Module): + def __init__(self, dim, dim_out=None, mult=4, glu=False, dropout=0.): + super().__init__() + inner_dim = int(dim * mult) + dim_out = default(dim_out, dim) + project_in = nn.Sequential( + nn.Linear(dim, inner_dim), + nn.GELU() + ) if not glu else GEGLU(dim, inner_dim) + + self.net = nn.Sequential( + project_in, + nn.Dropout(dropout), + nn.Linear(inner_dim, dim_out) + ) + + def forward(self, x): + return self.net(x) + + +def zero_module(module): + """ + Zero out the parameters of a module and return it. + """ + for p in module.parameters(): + p.detach().zero_() + return module + + +def Normalize(in_channels): + return torch.nn.GroupNorm(num_groups=32, num_channels=in_channels, eps=1e-6, affine=True) + + +class SpatialSelfAttention(nn.Module): + def __init__(self, in_channels): + super().__init__() + self.in_channels = in_channels + + self.norm = Normalize(in_channels) + self.q = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + self.k = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + self.v = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + self.proj_out = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + + def forward(self, x): + h_ = x + h_ = self.norm(h_) + q = self.q(h_) + k = self.k(h_) + v = self.v(h_) + + # compute attention + b,c,h,w = q.shape + q = rearrange(q, 'b c h w -> b (h w) c') + k = rearrange(k, 'b c h w -> b c (h w)') + w_ = torch.einsum('bij,bjk->bik', q, k) + + w_ = w_ * (int(c)**(-0.5)) + w_ = torch.nn.functional.softmax(w_, dim=2) + + # attend to values + v = rearrange(v, 'b c h w -> b c (h w)') + w_ = rearrange(w_, 'b i j -> b j i') + h_ = torch.einsum('bij,bjk->bik', v, w_) + h_ = rearrange(h_, 'b c (h w) -> b c h w', h=h) + h_ = self.proj_out(h_) + + return x+h_ + + +class CrossAttention(nn.Module): + def __init__(self, query_dim, context_dim=None, heads=8, dim_head=64, dropout=0.): + super().__init__() + inner_dim = dim_head * heads + context_dim = default(context_dim, query_dim) + + self.scale = dim_head ** -0.5 + self.heads = heads + + self.to_q = nn.Linear(query_dim, inner_dim, bias=False) + self.to_k = nn.Linear(context_dim, inner_dim, bias=False) + self.to_v = nn.Linear(context_dim, inner_dim, bias=False) + + self.to_out = nn.Sequential( + nn.Linear(inner_dim, query_dim), + nn.Dropout(dropout) + ) + + def forward(self, x, context=None, mask=None): + + h = self.heads + + # breakpoint() + q = self.to_q(x) + context = default(context, x) + k = self.to_k(context) + v = self.to_v(context) + # breakpoint() + if len(k.shape) == 2: + k = rearrange(k, 'b d -> b 1 d') + v = rearrange(v, 'b d -> b 1 d') + k = repeat(k, 'b 1 d -> b n d', n=q.shape[1]) + v = repeat(v, 'b 1 d -> b n d', n=q.shape[1]) + + q, k, v = map(lambda t: rearrange(t, 'b n (h d) -> (b h) n d', h=h), (q, k, v)) + + # force cast to fp32 to avoid overflowing + if _ATTN_PRECISION =="fp32": + with torch.autocast(enabled=False, device_type = 'cuda'): + q, k = q.float(), k.float() + sim = einsum('b i d, b j d -> b i j', q, k) * self.scale + else: + sim = einsum('b i d, b j d -> b i j', q, k) * self.scale + + del q, k + + if exists(mask): + mask = rearrange(mask, 'b ... -> b (...)') + max_neg_value = -torch.finfo(sim.dtype).max + mask = repeat(mask, 'b j -> (b h) () j', h=h) + sim.masked_fill_(~mask, max_neg_value) + + # attention, what we cannot get enough of + sim = sim.softmax(dim=-1) + + out = einsum('b i j, b j d -> b i d', sim, v) + out = rearrange(out, '(b h) n d -> b n (h d)', h=h) + return self.to_out(out) + + +class MemoryEfficientCrossAttention(nn.Module): + # https://github.com/MatthieuTPHR/diffusers/blob/d80b531ff8060ec1ea982b65a1b8df70f73aa67c/src/diffusers/models/attention.py#L223 + def __init__(self, query_dim, context_dim=None, heads=8, dim_head=64, dropout=0.0): + super().__init__() + print(f"Setting up {self.__class__.__name__}. Query dim is {query_dim}, context_dim is {context_dim} and using " + f"{heads} heads.") + inner_dim = dim_head * heads + context_dim = default(context_dim, query_dim) + + self.heads = heads + self.dim_head = dim_head + + self.to_q = nn.Linear(query_dim, inner_dim, bias=False) + self.to_k = nn.Linear(context_dim, inner_dim, bias=False) + self.to_v = nn.Linear(context_dim, inner_dim, bias=False) + + self.to_out = nn.Sequential(nn.Linear(inner_dim, query_dim), nn.Dropout(dropout)) + self.attention_op: Optional[Any] = None + + def forward(self, x, context=None, mask=None): + q = self.to_q(x) + context = default(context, x) + k = self.to_k(context) + v = self.to_v(context) + + b, _, _ = q.shape + q, k, v = map( + lambda t: t.unsqueeze(3) + .reshape(b, t.shape[1], self.heads, self.dim_head) + .permute(0, 2, 1, 3) + .reshape(b * self.heads, t.shape[1], self.dim_head) + .contiguous(), + (q, k, v), + ) + + # actually compute the attention, what we cannot get enough of + out = xformers.ops.memory_efficient_attention(q, k, v, attn_bias=None, op=self.attention_op) + + if exists(mask): + raise NotImplementedError + out = ( + out.unsqueeze(0) + .reshape(b, self.heads, out.shape[1], self.dim_head) + .permute(0, 2, 1, 3) + .reshape(b, out.shape[1], self.heads * self.dim_head) + ) + return self.to_out(out) + + +class BasicTransformerBlock(nn.Module): + ATTENTION_MODES = { + "softmax": CrossAttention, # vanilla attention + "softmax-xformers": MemoryEfficientCrossAttention + } + def __init__(self, dim, n_heads, d_head, dropout=0., context_dim=None, gated_ff=True, checkpoint=True, + disable_self_attn=False): + super().__init__() + attn_mode = "softmax-xformers" if XFORMERS_IS_AVAILBLE else "softmax" + # breakpoint() + assert attn_mode in self.ATTENTION_MODES + attn_cls = self.ATTENTION_MODES[attn_mode] + self.disable_self_attn = disable_self_attn + self.attn1 = attn_cls(query_dim=dim, heads=n_heads, dim_head=d_head, dropout=dropout, + context_dim=context_dim if self.disable_self_attn else None) # is a self-attention if not self.disable_self_attn + self.ff = FeedForward(dim, dropout=dropout, glu=gated_ff) + self.attn2 = attn_cls(query_dim=dim, context_dim=context_dim, + heads=n_heads, dim_head=d_head, dropout=dropout) # is self-attn if context is none + self.norm1 = nn.LayerNorm(dim) + self.norm2 = nn.LayerNorm(dim) + self.norm3 = nn.LayerNorm(dim) + self.checkpoint = checkpoint + + def forward(self, x, context=None): + return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint) + + def _forward(self, x, context=None): + x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None) + x + x = self.attn2(self.norm2(x), context=context) + x + x = self.ff(self.norm3(x)) + x + return x + + +class SpatialTransformer(nn.Module): + """ + Transformer block for image-like data. + First, project the input (aka embedding) + and reshape to b, t, d. + Then apply standard transformer action. + Finally, reshape to image + NEW: use_linear for more efficiency instead of the 1x1 convs + """ + def __init__(self, in_channels, n_heads, d_head, + depth=1, dropout=0., context_dim=None, + disable_self_attn=False, use_linear=False, + use_checkpoint=True): + super().__init__() + if exists(context_dim) and not isinstance(context_dim, list): + context_dim = [context_dim] + self.in_channels = in_channels + inner_dim = n_heads * d_head + self.norm = Normalize(in_channels) + if not use_linear: + self.proj_in = nn.Conv2d(in_channels, + inner_dim, + kernel_size=1, + stride=1, + padding=0) + else: + self.proj_in = nn.Linear(in_channels, inner_dim) + + self.transformer_blocks = nn.ModuleList( + [BasicTransformerBlock(inner_dim, n_heads, d_head, dropout=dropout, context_dim=context_dim[d], + disable_self_attn=disable_self_attn, checkpoint=use_checkpoint) + for d in range(depth)] + ) + if not use_linear: + self.proj_out = zero_module(nn.Conv2d(inner_dim, + in_channels, + kernel_size=1, + stride=1, + padding=0)) + else: + self.proj_out = zero_module(nn.Linear(in_channels, inner_dim)) + self.use_linear = use_linear + + def forward(self, x, context=None): + # note: if no context is given, cross-attention defaults to self-attention + if not isinstance(context, list): + context = [context] + b, c, h, w = x.shape + x_in = x + x = self.norm(x) + # breakpoint() + if not self.use_linear: + x = self.proj_in(x) + + x = rearrange(x, 'b c h w -> b (h w) c').contiguous() + # context = [rearrange(context[0], 'b c h w -> b (h w) c').contiguous()] + # breakpoint() + if self.use_linear: + x = self.proj_in(x) + for i, block in enumerate(self.transformer_blocks): + x = block(x, context=context[i]) + if self.use_linear: + x = self.proj_out(x) + x = rearrange(x, 'b (h w) c -> b c h w', h=h, w=w).contiguous() + if not self.use_linear: + x = self.proj_out(x) + return x + x_in + diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__init__.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..4a91126760361e82542e067e593f7aa9dd4bc028 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..a3f98f327cb504191d2abba95d9ae6feabfbea79 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..9957382e78468ef32a1d8741dbf6f3bf42201adf Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..aa2d42a94fa534c01ca6abf5f747e54d37ca270c Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/__init__.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..047bb162a269367fe90f84cd9338e287e6b553f5 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..a00765c71faa478cfe6ac5b1d05f650682cf1cb4 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..257ffc36147da91e109292741e4caa563f7e233b Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..dcd6faa168b915d295e78e404c1de79754c55fc4 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/model.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..4ab69d3c56c524e54ce72f82481898fdea92d086 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..ed99f829e16d5a22dc9860f11d31f1c2b89b31c5 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..6c0fadc0dd93950fba74d5f959b6d128d82a672c Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..80a37593ac7489363ab6092c9b341411c41893ba Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..41d8fb866d954d862d07b9df3ed0291e524660c6 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..abf2bfb8f352a95d08363657a505c2b8567f1847 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..5cab49e57fef32cf20896c3e43d4937b2a011cb8 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/openaimodel_pseudo3D.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..6488a22f45e90fc76c86f167067187aa2619af36 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..f2cec22050ed7397b0d9bd32b049f92ba4aa220d Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..b92fc5516110d956ef16841845c5ae5e47649b74 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..fb9e40fb32ac39c35b2b6a400cfe84bedd41c3ee Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/__pycache__/util.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/model.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/model.py new file mode 100644 index 0000000000000000000000000000000000000000..b089eebbe1676d8249005bb9def002ff5180715b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/model.py @@ -0,0 +1,852 @@ +# pytorch_diffusion + derived encoder decoder +import math +import torch +import torch.nn as nn +import numpy as np +from einops import rearrange +from typing import Optional, Any + +from ldm.modules.attention import MemoryEfficientCrossAttention + +try: + import xformers + import xformers.ops + XFORMERS_IS_AVAILBLE = True +except: + XFORMERS_IS_AVAILBLE = False + print("No module 'xformers'. Proceeding without it.") + + +def get_timestep_embedding(timesteps, embedding_dim): + """ + This matches the implementation in Denoising Diffusion Probabilistic Models: + From Fairseq. + Build sinusoidal embeddings. + This matches the implementation in tensor2tensor, but differs slightly + from the description in Section 3.5 of "Attention Is All You Need". + """ + assert len(timesteps.shape) == 1 + + half_dim = embedding_dim // 2 + emb = math.log(10000) / (half_dim - 1) + emb = torch.exp(torch.arange(half_dim, dtype=torch.float32) * -emb) + emb = emb.to(device=timesteps.device) + emb = timesteps.float()[:, None] * emb[None, :] + emb = torch.cat([torch.sin(emb), torch.cos(emb)], dim=1) + if embedding_dim % 2 == 1: # zero pad + emb = torch.nn.functional.pad(emb, (0,1,0,0)) + return emb + + +def nonlinearity(x): + # swish + return x*torch.sigmoid(x) + + +def Normalize(in_channels, num_groups=32): + return torch.nn.GroupNorm(num_groups=num_groups, num_channels=in_channels, eps=1e-6, affine=True) + + +class Upsample(nn.Module): + def __init__(self, in_channels, with_conv): + super().__init__() + self.with_conv = with_conv + if self.with_conv: + self.conv = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=3, + stride=1, + padding=1) + + def forward(self, x): + x = torch.nn.functional.interpolate(x, scale_factor=2.0, mode="nearest") + if self.with_conv: + x = self.conv(x) + return x + + +class Downsample(nn.Module): + def __init__(self, in_channels, with_conv): + super().__init__() + self.with_conv = with_conv + if self.with_conv: + # no asymmetric padding in torch conv, must do it ourselves + self.conv = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=3, + stride=2, + padding=0) + + def forward(self, x): + if self.with_conv: + pad = (0,1,0,1) + x = torch.nn.functional.pad(x, pad, mode="constant", value=0) + x = self.conv(x) + else: + x = torch.nn.functional.avg_pool2d(x, kernel_size=2, stride=2) + return x + + +class ResnetBlock(nn.Module): + def __init__(self, *, in_channels, out_channels=None, conv_shortcut=False, + dropout, temb_channels=512): + super().__init__() + self.in_channels = in_channels + out_channels = in_channels if out_channels is None else out_channels + self.out_channels = out_channels + self.use_conv_shortcut = conv_shortcut + + self.norm1 = Normalize(in_channels) + self.conv1 = torch.nn.Conv2d(in_channels, + out_channels, + kernel_size=3, + stride=1, + padding=1) + if temb_channels > 0: + self.temb_proj = torch.nn.Linear(temb_channels, + out_channels) + self.norm2 = Normalize(out_channels) + self.dropout = torch.nn.Dropout(dropout) + self.conv2 = torch.nn.Conv2d(out_channels, + out_channels, + kernel_size=3, + stride=1, + padding=1) + if self.in_channels != self.out_channels: + if self.use_conv_shortcut: + self.conv_shortcut = torch.nn.Conv2d(in_channels, + out_channels, + kernel_size=3, + stride=1, + padding=1) + else: + self.nin_shortcut = torch.nn.Conv2d(in_channels, + out_channels, + kernel_size=1, + stride=1, + padding=0) + + def forward(self, x, temb): + h = x + h = self.norm1(h) + h = nonlinearity(h) + h = self.conv1(h) + + if temb is not None: + h = h + self.temb_proj(nonlinearity(temb))[:,:,None,None] + + h = self.norm2(h) + h = nonlinearity(h) + h = self.dropout(h) + h = self.conv2(h) + + if self.in_channels != self.out_channels: + if self.use_conv_shortcut: + x = self.conv_shortcut(x) + else: + x = self.nin_shortcut(x) + + return x+h + + +class AttnBlock(nn.Module): + def __init__(self, in_channels): + super().__init__() + self.in_channels = in_channels + + self.norm = Normalize(in_channels) + self.q = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + self.k = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + self.v = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + self.proj_out = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + + def forward(self, x): + h_ = x + h_ = self.norm(h_) + q = self.q(h_) + k = self.k(h_) + v = self.v(h_) + + # compute attention + b,c,h,w = q.shape + q = q.reshape(b,c,h*w) + q = q.permute(0,2,1) # b,hw,c + k = k.reshape(b,c,h*w) # b,c,hw + w_ = torch.bmm(q,k) # b,hw,hw w[b,i,j]=sum_c q[b,i,c]k[b,c,j] + w_ = w_ * (int(c)**(-0.5)) + w_ = torch.nn.functional.softmax(w_, dim=2) + + # attend to values + v = v.reshape(b,c,h*w) + w_ = w_.permute(0,2,1) # b,hw,hw (first hw of k, second of q) + h_ = torch.bmm(v,w_) # b, c,hw (hw of q) h_[b,c,j] = sum_i v[b,c,i] w_[b,i,j] + h_ = h_.reshape(b,c,h,w) + + h_ = self.proj_out(h_) + + return x+h_ + +class MemoryEfficientAttnBlock(nn.Module): + """ + Uses xformers efficient implementation, + see https://github.com/MatthieuTPHR/diffusers/blob/d80b531ff8060ec1ea982b65a1b8df70f73aa67c/src/diffusers/models/attention.py#L223 + Note: this is a single-head self-attention operation + """ + # + def __init__(self, in_channels): + super().__init__() + self.in_channels = in_channels + + self.norm = Normalize(in_channels) + self.q = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + self.k = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + self.v = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + self.proj_out = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=1, + stride=1, + padding=0) + self.attention_op: Optional[Any] = None + + def forward(self, x): + h_ = x + h_ = self.norm(h_) + q = self.q(h_) + k = self.k(h_) + v = self.v(h_) + + # compute attention + B, C, H, W = q.shape + q, k, v = map(lambda x: rearrange(x, 'b c h w -> b (h w) c'), (q, k, v)) + + q, k, v = map( + lambda t: t.unsqueeze(3) + .reshape(B, t.shape[1], 1, C) + .permute(0, 2, 1, 3) + .reshape(B * 1, t.shape[1], C) + .contiguous(), + (q, k, v), + ) + out = xformers.ops.memory_efficient_attention(q, k, v, attn_bias=None, op=self.attention_op) + + out = ( + out.unsqueeze(0) + .reshape(B, 1, out.shape[1], C) + .permute(0, 2, 1, 3) + .reshape(B, out.shape[1], C) + ) + out = rearrange(out, 'b (h w) c -> b c h w', b=B, h=H, w=W, c=C) + out = self.proj_out(out) + return x+out + + +class MemoryEfficientCrossAttentionWrapper(MemoryEfficientCrossAttention): + def forward(self, x, context=None, mask=None): + b, c, h, w = x.shape + x = rearrange(x, 'b c h w -> b (h w) c') + out = super().forward(x, context=context, mask=mask) + out = rearrange(out, 'b (h w) c -> b c h w', h=h, w=w, c=c) + return x + out + + +def make_attn(in_channels, attn_type="vanilla", attn_kwargs=None): + assert attn_type in ["vanilla", "vanilla-xformers", "memory-efficient-cross-attn", "linear", "none"], f'attn_type {attn_type} unknown' + if XFORMERS_IS_AVAILBLE and attn_type == "vanilla": + attn_type = "vanilla-xformers" + print(f"making attention of type '{attn_type}' with {in_channels} in_channels") + if attn_type == "vanilla": + assert attn_kwargs is None + return AttnBlock(in_channels) + elif attn_type == "vanilla-xformers": + print(f"building MemoryEfficientAttnBlock with {in_channels} in_channels...") + return MemoryEfficientAttnBlock(in_channels) + elif type == "memory-efficient-cross-attn": + attn_kwargs["query_dim"] = in_channels + return MemoryEfficientCrossAttentionWrapper(**attn_kwargs) + elif attn_type == "none": + return nn.Identity(in_channels) + else: + raise NotImplementedError() + + +class Model(nn.Module): + def __init__(self, *, ch, out_ch, ch_mult=(1,2,4,8), num_res_blocks, + attn_resolutions, dropout=0.0, resamp_with_conv=True, in_channels, + resolution, use_timestep=True, use_linear_attn=False, attn_type="vanilla"): + super().__init__() + if use_linear_attn: attn_type = "linear" + self.ch = ch + self.temb_ch = self.ch*4 + self.num_resolutions = len(ch_mult) + self.num_res_blocks = num_res_blocks + self.resolution = resolution + self.in_channels = in_channels + + self.use_timestep = use_timestep + if self.use_timestep: + # timestep embedding + self.temb = nn.Module() + self.temb.dense = nn.ModuleList([ + torch.nn.Linear(self.ch, + self.temb_ch), + torch.nn.Linear(self.temb_ch, + self.temb_ch), + ]) + + # downsampling + self.conv_in = torch.nn.Conv2d(in_channels, + self.ch, + kernel_size=3, + stride=1, + padding=1) + + curr_res = resolution + in_ch_mult = (1,)+tuple(ch_mult) + self.down = nn.ModuleList() + for i_level in range(self.num_resolutions): + block = nn.ModuleList() + attn = nn.ModuleList() + block_in = ch*in_ch_mult[i_level] + block_out = ch*ch_mult[i_level] + for i_block in range(self.num_res_blocks): + block.append(ResnetBlock(in_channels=block_in, + out_channels=block_out, + temb_channels=self.temb_ch, + dropout=dropout)) + block_in = block_out + if curr_res in attn_resolutions: + attn.append(make_attn(block_in, attn_type=attn_type)) + down = nn.Module() + down.block = block + down.attn = attn + if i_level != self.num_resolutions-1: + down.downsample = Downsample(block_in, resamp_with_conv) + curr_res = curr_res // 2 + self.down.append(down) + + # middle + self.mid = nn.Module() + self.mid.block_1 = ResnetBlock(in_channels=block_in, + out_channels=block_in, + temb_channels=self.temb_ch, + dropout=dropout) + self.mid.attn_1 = make_attn(block_in, attn_type=attn_type) + self.mid.block_2 = ResnetBlock(in_channels=block_in, + out_channels=block_in, + temb_channels=self.temb_ch, + dropout=dropout) + + # upsampling + self.up = nn.ModuleList() + for i_level in reversed(range(self.num_resolutions)): + block = nn.ModuleList() + attn = nn.ModuleList() + block_out = ch*ch_mult[i_level] + skip_in = ch*ch_mult[i_level] + for i_block in range(self.num_res_blocks+1): + if i_block == self.num_res_blocks: + skip_in = ch*in_ch_mult[i_level] + block.append(ResnetBlock(in_channels=block_in+skip_in, + out_channels=block_out, + temb_channels=self.temb_ch, + dropout=dropout)) + block_in = block_out + if curr_res in attn_resolutions: + attn.append(make_attn(block_in, attn_type=attn_type)) + up = nn.Module() + up.block = block + up.attn = attn + if i_level != 0: + up.upsample = Upsample(block_in, resamp_with_conv) + curr_res = curr_res * 2 + self.up.insert(0, up) # prepend to get consistent order + + # end + self.norm_out = Normalize(block_in) + self.conv_out = torch.nn.Conv2d(block_in, + out_ch, + kernel_size=3, + stride=1, + padding=1) + + def forward(self, x, t=None, context=None): + #assert x.shape[2] == x.shape[3] == self.resolution + if context is not None: + # assume aligned context, cat along channel axis + x = torch.cat((x, context), dim=1) + if self.use_timestep: + # timestep embedding + assert t is not None + temb = get_timestep_embedding(t, self.ch) + temb = self.temb.dense[0](temb) + temb = nonlinearity(temb) + temb = self.temb.dense[1](temb) + else: + temb = None + + # downsampling + hs = [self.conv_in(x)] + for i_level in range(self.num_resolutions): + for i_block in range(self.num_res_blocks): + h = self.down[i_level].block[i_block](hs[-1], temb) + if len(self.down[i_level].attn) > 0: + h = self.down[i_level].attn[i_block](h) + hs.append(h) + if i_level != self.num_resolutions-1: + hs.append(self.down[i_level].downsample(hs[-1])) + + # middle + h = hs[-1] + h = self.mid.block_1(h, temb) + h = self.mid.attn_1(h) + h = self.mid.block_2(h, temb) + + # upsampling + for i_level in reversed(range(self.num_resolutions)): + for i_block in range(self.num_res_blocks+1): + h = self.up[i_level].block[i_block]( + torch.cat([h, hs.pop()], dim=1), temb) + if len(self.up[i_level].attn) > 0: + h = self.up[i_level].attn[i_block](h) + if i_level != 0: + h = self.up[i_level].upsample(h) + + # end + h = self.norm_out(h) + h = nonlinearity(h) + h = self.conv_out(h) + return h + + def get_last_layer(self): + return self.conv_out.weight + + +class Encoder(nn.Module): + def __init__(self, *, ch, out_ch, ch_mult=(1,2,4,8), num_res_blocks, + attn_resolutions, dropout=0.0, resamp_with_conv=True, in_channels, + resolution, z_channels, double_z=True, use_linear_attn=False, attn_type="vanilla", + **ignore_kwargs): + super().__init__() + if use_linear_attn: attn_type = "linear" + self.ch = ch + self.temb_ch = 0 + self.num_resolutions = len(ch_mult) + self.num_res_blocks = num_res_blocks + self.resolution = resolution + self.in_channels = in_channels + + # downsampling + self.conv_in = torch.nn.Conv2d(in_channels, + self.ch, + kernel_size=3, + stride=1, + padding=1) + + curr_res = resolution + in_ch_mult = (1,)+tuple(ch_mult) + self.in_ch_mult = in_ch_mult + self.down = nn.ModuleList() + for i_level in range(self.num_resolutions): + block = nn.ModuleList() + attn = nn.ModuleList() + block_in = ch*in_ch_mult[i_level] + block_out = ch*ch_mult[i_level] + for i_block in range(self.num_res_blocks): + block.append(ResnetBlock(in_channels=block_in, + out_channels=block_out, + temb_channels=self.temb_ch, + dropout=dropout)) + block_in = block_out + if curr_res in attn_resolutions: + attn.append(make_attn(block_in, attn_type=attn_type)) + down = nn.Module() + down.block = block + down.attn = attn + if i_level != self.num_resolutions-1: + down.downsample = Downsample(block_in, resamp_with_conv) + curr_res = curr_res // 2 + self.down.append(down) + + # middle + self.mid = nn.Module() + self.mid.block_1 = ResnetBlock(in_channels=block_in, + out_channels=block_in, + temb_channels=self.temb_ch, + dropout=dropout) + self.mid.attn_1 = make_attn(block_in, attn_type=attn_type) + self.mid.block_2 = ResnetBlock(in_channels=block_in, + out_channels=block_in, + temb_channels=self.temb_ch, + dropout=dropout) + + # end + self.norm_out = Normalize(block_in) + self.conv_out = torch.nn.Conv2d(block_in, + 2*z_channels if double_z else z_channels, + kernel_size=3, + stride=1, + padding=1) + + def forward(self, x): + # timestep embedding + temb = None + + # downsampling + hs = [self.conv_in(x)] + for i_level in range(self.num_resolutions): + for i_block in range(self.num_res_blocks): + h = self.down[i_level].block[i_block](hs[-1], temb) + if len(self.down[i_level].attn) > 0: + h = self.down[i_level].attn[i_block](h) + hs.append(h) + if i_level != self.num_resolutions-1: + hs.append(self.down[i_level].downsample(hs[-1])) + + # middle + h = hs[-1] + h = self.mid.block_1(h, temb) + h = self.mid.attn_1(h) + h = self.mid.block_2(h, temb) + + # end + h = self.norm_out(h) + h = nonlinearity(h) + h = self.conv_out(h) + return h + + +class Decoder(nn.Module): + def __init__(self, *, ch, out_ch, ch_mult=(1,2,4,8), num_res_blocks, + attn_resolutions, dropout=0.0, resamp_with_conv=True, in_channels, + resolution, z_channels, give_pre_end=False, tanh_out=False, use_linear_attn=False, + attn_type="vanilla", **ignorekwargs): + super().__init__() + if use_linear_attn: attn_type = "linear" + self.ch = ch + self.temb_ch = 0 + self.num_resolutions = len(ch_mult) + self.num_res_blocks = num_res_blocks + self.resolution = resolution + self.in_channels = in_channels + self.give_pre_end = give_pre_end + self.tanh_out = tanh_out + + # compute in_ch_mult, block_in and curr_res at lowest res + in_ch_mult = (1,)+tuple(ch_mult) + block_in = ch*ch_mult[self.num_resolutions-1] + curr_res = resolution // 2**(self.num_resolutions-1) + self.z_shape = (1,z_channels,curr_res,curr_res) + print("Working with z of shape {} = {} dimensions.".format( + self.z_shape, np.prod(self.z_shape))) + + # z to block_in + self.conv_in = torch.nn.Conv2d(z_channels, + block_in, + kernel_size=3, + stride=1, + padding=1) + + # middle + self.mid = nn.Module() + self.mid.block_1 = ResnetBlock(in_channels=block_in, + out_channels=block_in, + temb_channels=self.temb_ch, + dropout=dropout) + self.mid.attn_1 = make_attn(block_in, attn_type=attn_type) + self.mid.block_2 = ResnetBlock(in_channels=block_in, + out_channels=block_in, + temb_channels=self.temb_ch, + dropout=dropout) + + # upsampling + self.up = nn.ModuleList() + for i_level in reversed(range(self.num_resolutions)): + block = nn.ModuleList() + attn = nn.ModuleList() + block_out = ch*ch_mult[i_level] + for i_block in range(self.num_res_blocks+1): + block.append(ResnetBlock(in_channels=block_in, + out_channels=block_out, + temb_channels=self.temb_ch, + dropout=dropout)) + block_in = block_out + if curr_res in attn_resolutions: + attn.append(make_attn(block_in, attn_type=attn_type)) + up = nn.Module() + up.block = block + up.attn = attn + if i_level != 0: + up.upsample = Upsample(block_in, resamp_with_conv) + curr_res = curr_res * 2 + self.up.insert(0, up) # prepend to get consistent order + + # end + self.norm_out = Normalize(block_in) + self.conv_out = torch.nn.Conv2d(block_in, + out_ch, + kernel_size=3, + stride=1, + padding=1) + + def forward(self, z): + #assert z.shape[1:] == self.z_shape[1:] + self.last_z_shape = z.shape + + # timestep embedding + temb = None + + # z to block_in + h = self.conv_in(z) + + # middle + h = self.mid.block_1(h, temb) + h = self.mid.attn_1(h) + h = self.mid.block_2(h, temb) + + # upsampling + for i_level in reversed(range(self.num_resolutions)): + for i_block in range(self.num_res_blocks+1): + h = self.up[i_level].block[i_block](h, temb) + if len(self.up[i_level].attn) > 0: + h = self.up[i_level].attn[i_block](h) + if i_level != 0: + h = self.up[i_level].upsample(h) + + # end + if self.give_pre_end: + return h + + h = self.norm_out(h) + h = nonlinearity(h) + h = self.conv_out(h) + if self.tanh_out: + h = torch.tanh(h) + return h + + +class SimpleDecoder(nn.Module): + def __init__(self, in_channels, out_channels, *args, **kwargs): + super().__init__() + self.model = nn.ModuleList([nn.Conv2d(in_channels, in_channels, 1), + ResnetBlock(in_channels=in_channels, + out_channels=2 * in_channels, + temb_channels=0, dropout=0.0), + ResnetBlock(in_channels=2 * in_channels, + out_channels=4 * in_channels, + temb_channels=0, dropout=0.0), + ResnetBlock(in_channels=4 * in_channels, + out_channels=2 * in_channels, + temb_channels=0, dropout=0.0), + nn.Conv2d(2*in_channels, in_channels, 1), + Upsample(in_channels, with_conv=True)]) + # end + self.norm_out = Normalize(in_channels) + self.conv_out = torch.nn.Conv2d(in_channels, + out_channels, + kernel_size=3, + stride=1, + padding=1) + + def forward(self, x): + for i, layer in enumerate(self.model): + if i in [1,2,3]: + x = layer(x, None) + else: + x = layer(x) + + h = self.norm_out(x) + h = nonlinearity(h) + x = self.conv_out(h) + return x + + +class UpsampleDecoder(nn.Module): + def __init__(self, in_channels, out_channels, ch, num_res_blocks, resolution, + ch_mult=(2,2), dropout=0.0): + super().__init__() + # upsampling + self.temb_ch = 0 + self.num_resolutions = len(ch_mult) + self.num_res_blocks = num_res_blocks + block_in = in_channels + curr_res = resolution // 2 ** (self.num_resolutions - 1) + self.res_blocks = nn.ModuleList() + self.upsample_blocks = nn.ModuleList() + for i_level in range(self.num_resolutions): + res_block = [] + block_out = ch * ch_mult[i_level] + for i_block in range(self.num_res_blocks + 1): + res_block.append(ResnetBlock(in_channels=block_in, + out_channels=block_out, + temb_channels=self.temb_ch, + dropout=dropout)) + block_in = block_out + self.res_blocks.append(nn.ModuleList(res_block)) + if i_level != self.num_resolutions - 1: + self.upsample_blocks.append(Upsample(block_in, True)) + curr_res = curr_res * 2 + + # end + self.norm_out = Normalize(block_in) + self.conv_out = torch.nn.Conv2d(block_in, + out_channels, + kernel_size=3, + stride=1, + padding=1) + + def forward(self, x): + # upsampling + h = x + for k, i_level in enumerate(range(self.num_resolutions)): + for i_block in range(self.num_res_blocks + 1): + h = self.res_blocks[i_level][i_block](h, None) + if i_level != self.num_resolutions - 1: + h = self.upsample_blocks[k](h) + h = self.norm_out(h) + h = nonlinearity(h) + h = self.conv_out(h) + return h + + +class LatentRescaler(nn.Module): + def __init__(self, factor, in_channels, mid_channels, out_channels, depth=2): + super().__init__() + # residual block, interpolate, residual block + self.factor = factor + self.conv_in = nn.Conv2d(in_channels, + mid_channels, + kernel_size=3, + stride=1, + padding=1) + self.res_block1 = nn.ModuleList([ResnetBlock(in_channels=mid_channels, + out_channels=mid_channels, + temb_channels=0, + dropout=0.0) for _ in range(depth)]) + self.attn = AttnBlock(mid_channels) + self.res_block2 = nn.ModuleList([ResnetBlock(in_channels=mid_channels, + out_channels=mid_channels, + temb_channels=0, + dropout=0.0) for _ in range(depth)]) + + self.conv_out = nn.Conv2d(mid_channels, + out_channels, + kernel_size=1, + ) + + def forward(self, x): + x = self.conv_in(x) + for block in self.res_block1: + x = block(x, None) + x = torch.nn.functional.interpolate(x, size=(int(round(x.shape[2]*self.factor)), int(round(x.shape[3]*self.factor)))) + x = self.attn(x) + for block in self.res_block2: + x = block(x, None) + x = self.conv_out(x) + return x + + +class MergedRescaleEncoder(nn.Module): + def __init__(self, in_channels, ch, resolution, out_ch, num_res_blocks, + attn_resolutions, dropout=0.0, resamp_with_conv=True, + ch_mult=(1,2,4,8), rescale_factor=1.0, rescale_module_depth=1): + super().__init__() + intermediate_chn = ch * ch_mult[-1] + self.encoder = Encoder(in_channels=in_channels, num_res_blocks=num_res_blocks, ch=ch, ch_mult=ch_mult, + z_channels=intermediate_chn, double_z=False, resolution=resolution, + attn_resolutions=attn_resolutions, dropout=dropout, resamp_with_conv=resamp_with_conv, + out_ch=None) + self.rescaler = LatentRescaler(factor=rescale_factor, in_channels=intermediate_chn, + mid_channels=intermediate_chn, out_channels=out_ch, depth=rescale_module_depth) + + def forward(self, x): + x = self.encoder(x) + x = self.rescaler(x) + return x + + +class MergedRescaleDecoder(nn.Module): + def __init__(self, z_channels, out_ch, resolution, num_res_blocks, attn_resolutions, ch, ch_mult=(1,2,4,8), + dropout=0.0, resamp_with_conv=True, rescale_factor=1.0, rescale_module_depth=1): + super().__init__() + tmp_chn = z_channels*ch_mult[-1] + self.decoder = Decoder(out_ch=out_ch, z_channels=tmp_chn, attn_resolutions=attn_resolutions, dropout=dropout, + resamp_with_conv=resamp_with_conv, in_channels=None, num_res_blocks=num_res_blocks, + ch_mult=ch_mult, resolution=resolution, ch=ch) + self.rescaler = LatentRescaler(factor=rescale_factor, in_channels=z_channels, mid_channels=tmp_chn, + out_channels=tmp_chn, depth=rescale_module_depth) + + def forward(self, x): + x = self.rescaler(x) + x = self.decoder(x) + return x + + +class Upsampler(nn.Module): + def __init__(self, in_size, out_size, in_channels, out_channels, ch_mult=2): + super().__init__() + assert out_size >= in_size + num_blocks = int(np.log2(out_size//in_size))+1 + factor_up = 1.+ (out_size % in_size) + print(f"Building {self.__class__.__name__} with in_size: {in_size} --> out_size {out_size} and factor {factor_up}") + self.rescaler = LatentRescaler(factor=factor_up, in_channels=in_channels, mid_channels=2*in_channels, + out_channels=in_channels) + self.decoder = Decoder(out_ch=out_channels, resolution=out_size, z_channels=in_channels, num_res_blocks=2, + attn_resolutions=[], in_channels=None, ch=in_channels, + ch_mult=[ch_mult for _ in range(num_blocks)]) + + def forward(self, x): + x = self.rescaler(x) + x = self.decoder(x) + return x + + +class Resize(nn.Module): + def __init__(self, in_channels=None, learned=False, mode="bilinear"): + super().__init__() + self.with_conv = learned + self.mode = mode + if self.with_conv: + print(f"Note: {self.__class__.__name} uses learned downsampling and will ignore the fixed {mode} mode") + raise NotImplementedError() + assert in_channels is not None + # no asymmetric padding in torch conv, must do it ourselves + self.conv = torch.nn.Conv2d(in_channels, + in_channels, + kernel_size=4, + stride=2, + padding=1) + + def forward(self, x, scale_factor=1.0): + if scale_factor==1.0: + return x + else: + x = torch.nn.functional.interpolate(x, mode=self.mode, align_corners=False, scale_factor=scale_factor) + return x diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/openaimodel.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/openaimodel.py new file mode 100644 index 0000000000000000000000000000000000000000..af442ac6b189aa0cc558126373e7a104afd31556 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/openaimodel.py @@ -0,0 +1,788 @@ +from abc import abstractmethod +import math + +import numpy as np +import torch as th +import torch.nn as nn +import torch.nn.functional as F + +from ldm.modules.diffusionmodules.util import ( + checkpoint, + conv_nd, + linear, + avg_pool_nd, + zero_module, + normalization, + timestep_embedding, +) +from ldm.modules.attention import SpatialTransformer +from ldm.util import exists + + +# dummy replace +def convert_module_to_f16(x): + pass + +def convert_module_to_f32(x): + pass + + +## go +class AttentionPool2d(nn.Module): + """ + Adapted from CLIP: https://github.com/openai/CLIP/blob/main/clip/model.py + """ + + def __init__( + self, + spacial_dim: int, + embed_dim: int, + num_heads_channels: int, + output_dim: int = None, + ): + super().__init__() + self.positional_embedding = nn.Parameter(th.randn(embed_dim, spacial_dim ** 2 + 1) / embed_dim ** 0.5) + self.qkv_proj = conv_nd(1, embed_dim, 3 * embed_dim, 1) + self.c_proj = conv_nd(1, embed_dim, output_dim or embed_dim, 1) + self.num_heads = embed_dim // num_heads_channels + self.attention = QKVAttention(self.num_heads) + + def forward(self, x): + b, c, *_spatial = x.shape + x = x.reshape(b, c, -1) # NC(HW) + x = th.cat([x.mean(dim=-1, keepdim=True), x], dim=-1) # NC(HW+1) + x = x + self.positional_embedding[None, :, :].to(x.dtype) # NC(HW+1) + x = self.qkv_proj(x) + x = self.attention(x) + x = self.c_proj(x) + return x[:, :, 0] + + +class TimestepBlock(nn.Module): + """ + Any module where forward() takes timestep embeddings as a second argument. + """ + + @abstractmethod + def forward(self, x, emb): + """ + Apply the module to `x` given `emb` timestep embeddings. + """ + + +class TimestepEmbedSequential(nn.Sequential, TimestepBlock): + """ + A sequential module that passes timestep embeddings to the children that + support it as an extra input. + """ + + def forward(self, x, emb, context=None): + for layer in self: + if isinstance(layer, TimestepBlock): + x = layer(x, emb) + elif isinstance(layer, SpatialTransformer): + x = layer(x, context) + else: + x = layer(x) + return x + + +class Upsample(nn.Module): + """ + An upsampling layer with an optional convolution. + :param channels: channels in the inputs and outputs. + :param use_conv: a bool determining if a convolution is applied. + :param dims: determines if the signal is 1D, 2D, or 3D. If 3D, then + upsampling occurs in the inner-two dimensions. + """ + + def __init__(self, channels, use_conv, dims=2, out_channels=None, padding=1): + super().__init__() + self.channels = channels + self.out_channels = out_channels or channels + self.use_conv = use_conv + self.dims = dims + if use_conv: + self.conv = conv_nd(dims, self.channels, self.out_channels, 3, padding=padding) + + def forward(self, x): + assert x.shape[1] == self.channels + if self.dims == 3: + x = F.interpolate( + x, (x.shape[2], x.shape[3] * 2, x.shape[4] * 2), mode="nearest" + ) + else: + x = F.interpolate(x, scale_factor=2, mode="nearest") + if self.use_conv: + x = self.conv(x) + return x + +class TransposedUpsample(nn.Module): + 'Learned 2x upsampling without padding' + def __init__(self, channels, out_channels=None, ks=5): + super().__init__() + self.channels = channels + self.out_channels = out_channels or channels + + self.up = nn.ConvTranspose2d(self.channels,self.out_channels,kernel_size=ks,stride=2) + + def forward(self,x): + return self.up(x) + + +class Downsample(nn.Module): + """ + A downsampling layer with an optional convolution. + :param channels: channels in the inputs and outputs. + :param use_conv: a bool determining if a convolution is applied. + :param dims: determines if the signal is 1D, 2D, or 3D. If 3D, then + downsampling occurs in the inner-two dimensions. + """ + + def __init__(self, channels, use_conv, dims=2, out_channels=None,padding=1): + super().__init__() + self.channels = channels + self.out_channels = out_channels or channels + self.use_conv = use_conv + self.dims = dims + stride = 2 if dims != 3 else (1, 2, 2) + if use_conv: + self.op = conv_nd( + dims, self.channels, self.out_channels, 3, stride=stride, padding=padding + ) + else: + assert self.channels == self.out_channels + self.op = avg_pool_nd(dims, kernel_size=stride, stride=stride) + + def forward(self, x): + assert x.shape[1] == self.channels + return self.op(x) + + +class ResBlock(TimestepBlock): + """ + A residual block that can optionally change the number of channels. + :param channels: the number of input channels. + :param emb_channels: the number of timestep embedding channels. + :param dropout: the rate of dropout. + :param out_channels: if specified, the number of out channels. + :param use_conv: if True and out_channels is specified, use a spatial + convolution instead of a smaller 1x1 convolution to change the + channels in the skip connection. + :param dims: determines if the signal is 1D, 2D, or 3D. + :param use_checkpoint: if True, use gradient checkpointing on this module. + :param up: if True, use this block for upsampling. + :param down: if True, use this block for downsampling. + """ + + def __init__( + self, + channels, + emb_channels, + dropout, + out_channels=None, + use_conv=False, + use_scale_shift_norm=False, + dims=2, + use_checkpoint=False, + up=False, + down=False, + ): + super().__init__() + self.channels = channels + self.emb_channels = emb_channels + self.dropout = dropout + self.out_channels = out_channels or channels + self.use_conv = use_conv + self.use_checkpoint = use_checkpoint + self.use_scale_shift_norm = use_scale_shift_norm + + self.in_layers = nn.Sequential( + normalization(channels), + nn.SiLU(), + conv_nd(dims, channels, self.out_channels, 3, padding=1), + ) + + self.updown = up or down + + if up: + self.h_upd = Upsample(channels, False, dims) + self.x_upd = Upsample(channels, False, dims) + elif down: + self.h_upd = Downsample(channels, False, dims) + self.x_upd = Downsample(channels, False, dims) + else: + self.h_upd = self.x_upd = nn.Identity() + + self.emb_layers = nn.Sequential( + nn.SiLU(), + linear( + emb_channels, + 2 * self.out_channels if use_scale_shift_norm else self.out_channels, + ), + ) + self.out_layers = nn.Sequential( + normalization(self.out_channels), + nn.SiLU(), + nn.Dropout(p=dropout), + zero_module( + conv_nd(dims, self.out_channels, self.out_channels, 3, padding=1) + ), + ) + + if self.out_channels == channels: + self.skip_connection = nn.Identity() + elif use_conv: + self.skip_connection = conv_nd( + dims, channels, self.out_channels, 3, padding=1 + ) + else: + self.skip_connection = conv_nd(dims, channels, self.out_channels, 1) + + def forward(self, x, emb): + """ + Apply the block to a Tensor, conditioned on a timestep embedding. + :param x: an [N x C x ...] Tensor of features. + :param emb: an [N x emb_channels] Tensor of timestep embeddings. + :return: an [N x C x ...] Tensor of outputs. + """ + return checkpoint( + self._forward, (x, emb), self.parameters(), self.use_checkpoint + ) + + + def _forward(self, x, emb): + if self.updown: + in_rest, in_conv = self.in_layers[:-1], self.in_layers[-1] + h = in_rest(x) + h = self.h_upd(h) + x = self.x_upd(x) + h = in_conv(h) + else: + h = self.in_layers(x) + emb_out = self.emb_layers(emb).type(h.dtype) + while len(emb_out.shape) < len(h.shape): + emb_out = emb_out[..., None] + if self.use_scale_shift_norm: + out_norm, out_rest = self.out_layers[0], self.out_layers[1:] + scale, shift = th.chunk(emb_out, 2, dim=1) + h = out_norm(h) * (1 + scale) + shift + h = out_rest(h) + else: + h = h + emb_out + h = self.out_layers(h) + return self.skip_connection(x) + h + + +class AttentionBlock(nn.Module): + """ + An attention block that allows spatial positions to attend to each other. + Originally ported from here, but adapted to the N-d case. + https://github.com/hojonathanho/diffusion/blob/1e0dceb3b3495bbe19116a5e1b3596cd0706c543/diffusion_tf/models/unet.py#L66. + """ + + def __init__( + self, + channels, + num_heads=1, + num_head_channels=-1, + use_checkpoint=False, + use_new_attention_order=False, + ): + super().__init__() + self.channels = channels + if num_head_channels == -1: + self.num_heads = num_heads + else: + assert ( + channels % num_head_channels == 0 + ), f"q,k,v channels {channels} is not divisible by num_head_channels {num_head_channels}" + self.num_heads = channels // num_head_channels + self.use_checkpoint = use_checkpoint + self.norm = normalization(channels) + self.qkv = conv_nd(1, channels, channels * 3, 1) + if use_new_attention_order: + # split qkv before split heads + self.attention = QKVAttention(self.num_heads) + else: + # split heads before split qkv + self.attention = QKVAttentionLegacy(self.num_heads) + + self.proj_out = zero_module(conv_nd(1, channels, channels, 1)) + + def forward(self, x): + return checkpoint(self._forward, (x,), self.parameters(), True) # TODO: check checkpoint usage, is True # TODO: fix the .half call!!! + #return pt_checkpoint(self._forward, x) # pytorch + + def _forward(self, x): + b, c, *spatial = x.shape + x = x.reshape(b, c, -1) + qkv = self.qkv(self.norm(x)) + h = self.attention(qkv) + h = self.proj_out(h) + return (x + h).reshape(b, c, *spatial) + + +def count_flops_attn(model, _x, y): + """ + A counter for the `thop` package to count the operations in an + attention operation. + Meant to be used like: + macs, params = thop.profile( + model, + inputs=(inputs, timestamps), + custom_ops={QKVAttention: QKVAttention.count_flops}, + ) + """ + b, c, *spatial = y[0].shape + num_spatial = int(np.prod(spatial)) + # We perform two matmuls with the same number of ops. + # The first computes the weight matrix, the second computes + # the combination of the value vectors. + matmul_ops = 2 * b * (num_spatial ** 2) * c + model.total_ops += th.DoubleTensor([matmul_ops]) + + +class QKVAttentionLegacy(nn.Module): + """ + A module which performs QKV attention. Matches legacy QKVAttention + input/ouput heads shaping + """ + + def __init__(self, n_heads): + super().__init__() + self.n_heads = n_heads + + def forward(self, qkv): + """ + Apply QKV attention. + :param qkv: an [N x (H * 3 * C) x T] tensor of Qs, Ks, and Vs. + :return: an [N x (H * C) x T] tensor after attention. + """ + bs, width, length = qkv.shape + assert width % (3 * self.n_heads) == 0 + ch = width // (3 * self.n_heads) + q, k, v = qkv.reshape(bs * self.n_heads, ch * 3, length).split(ch, dim=1) + scale = 1 / math.sqrt(math.sqrt(ch)) + weight = th.einsum( + "bct,bcs->bts", q * scale, k * scale + ) # More stable with f16 than dividing afterwards + weight = th.softmax(weight.float(), dim=-1).type(weight.dtype) + a = th.einsum("bts,bcs->bct", weight, v) + return a.reshape(bs, -1, length) + + @staticmethod + def count_flops(model, _x, y): + return count_flops_attn(model, _x, y) + + +class QKVAttention(nn.Module): + """ + A module which performs QKV attention and splits in a different order. + """ + + def __init__(self, n_heads): + super().__init__() + self.n_heads = n_heads + + def forward(self, qkv): + """ + Apply QKV attention. + :param qkv: an [N x (3 * H * C) x T] tensor of Qs, Ks, and Vs. + :return: an [N x (H * C) x T] tensor after attention. + """ + bs, width, length = qkv.shape + assert width % (3 * self.n_heads) == 0 + ch = width // (3 * self.n_heads) + q, k, v = qkv.chunk(3, dim=1) + scale = 1 / math.sqrt(math.sqrt(ch)) + weight = th.einsum( + "bct,bcs->bts", + (q * scale).view(bs * self.n_heads, ch, length), + (k * scale).view(bs * self.n_heads, ch, length), + ) # More stable with f16 than dividing afterwards + weight = th.softmax(weight.float(), dim=-1).type(weight.dtype) + a = th.einsum("bts,bcs->bct", weight, v.reshape(bs * self.n_heads, ch, length)) + return a.reshape(bs, -1, length) + + @staticmethod + def count_flops(model, _x, y): + return count_flops_attn(model, _x, y) + + +class UNetModel(nn.Module): + """ + The full UNet model with attention and timestep embedding. + :param in_channels: channels in the input Tensor. + :param model_channels: base channel count for the model. + :param out_channels: channels in the output Tensor. + :param num_res_blocks: number of residual blocks per downsample. + :param attention_resolutions: a collection of downsample rates at which + attention will take place. May be a set, list, or tuple. + For example, if this contains 4, then at 4x downsampling, attention + will be used. + :param dropout: the dropout probability. + :param channel_mult: channel multiplier for each level of the UNet. + :param conv_resample: if True, use learned convolutions for upsampling and + downsampling. + :param dims: determines if the signal is 1D, 2D, or 3D. + :param num_classes: if specified (as an int), then this model will be + class-conditional with `num_classes` classes. + :param use_checkpoint: use gradient checkpointing to reduce memory usage. + :param num_heads: the number of attention heads in each attention layer. + :param num_heads_channels: if specified, ignore num_heads and instead use + a fixed channel width per attention head. + :param num_heads_upsample: works with num_heads to set a different number + of heads for upsampling. Deprecated. + :param use_scale_shift_norm: use a FiLM-like conditioning mechanism. + :param resblock_updown: use residual blocks for up/downsampling. + :param use_new_attention_order: use a different attention pattern for potentially + increased efficiency. + """ + + def __init__( + self, + image_size, + in_channels, + model_channels, + out_channels, + num_res_blocks, + attention_resolutions, + dropout=0, + channel_mult=(1, 2, 4, 8), + conv_resample=True, + dims=2, + num_classes=None, + use_checkpoint=False, + use_fp16=False, + num_heads=-1, + num_head_channels=-1, + num_heads_upsample=-1, + use_scale_shift_norm=False, + resblock_updown=False, + use_new_attention_order=False, + use_spatial_transformer=False, # custom transformer support + transformer_depth=1, # custom transformer support + context_dim=None, # custom transformer support + n_embed=None, # custom support for prediction of discrete ids into codebook of first stage vq model + legacy=True, + disable_self_attentions=None, + num_attention_blocks=None, + disable_middle_self_attn=False, + use_linear_in_transformer=False, + ): + super().__init__() + if use_spatial_transformer: + assert context_dim is not None, 'Fool!! You forgot to include the dimension of your cross-attention conditioning...' + + if context_dim is not None: + assert use_spatial_transformer, 'Fool!! You forgot to use the spatial transformer for your cross-attention conditioning...' + from omegaconf.listconfig import ListConfig + if type(context_dim) == ListConfig: + context_dim = list(context_dim) + + if num_heads_upsample == -1: + num_heads_upsample = num_heads + + if num_heads == -1: + assert num_head_channels != -1, 'Either num_heads or num_head_channels has to be set' + + if num_head_channels == -1: + assert num_heads != -1, 'Either num_heads or num_head_channels has to be set' + + self.image_size = image_size + self.in_channels = in_channels + self.model_channels = model_channels + self.out_channels = out_channels + if isinstance(num_res_blocks, int): + self.num_res_blocks = len(channel_mult) * [num_res_blocks] + else: + if len(num_res_blocks) != len(channel_mult): + raise ValueError("provide num_res_blocks either as an int (globally constant) or " + "as a list/tuple (per-level) with the same length as channel_mult") + self.num_res_blocks = num_res_blocks + if disable_self_attentions is not None: + # should be a list of booleans, indicating whether to disable self-attention in TransformerBlocks or not + assert len(disable_self_attentions) == len(channel_mult) + if num_attention_blocks is not None: + assert len(num_attention_blocks) == len(self.num_res_blocks) + assert all(map(lambda i: self.num_res_blocks[i] >= num_attention_blocks[i], range(len(num_attention_blocks)))) + print(f"Constructor of UNetModel received num_attention_blocks={num_attention_blocks}. " + f"This option has LESS priority than attention_resolutions {attention_resolutions}, " + f"i.e., in cases where num_attention_blocks[i] > 0 but 2**i not in attention_resolutions, " + f"attention will still not be set.") + + self.attention_resolutions = attention_resolutions + self.dropout = dropout + self.channel_mult = channel_mult + self.conv_resample = conv_resample + self.num_classes = num_classes + self.use_checkpoint = use_checkpoint + self.dtype = th.float16 if use_fp16 else th.float32 + self.num_heads = num_heads + self.num_head_channels = num_head_channels + self.num_heads_upsample = num_heads_upsample + self.predict_codebook_ids = n_embed is not None + + time_embed_dim = model_channels * 4 + self.time_embed = nn.Sequential( + linear(model_channels, time_embed_dim), + nn.SiLU(), + linear(time_embed_dim, time_embed_dim), + ) + + if self.num_classes is not None: + if isinstance(self.num_classes, int): + self.label_emb = nn.Embedding(num_classes, time_embed_dim) + elif self.num_classes == "continuous": + print("setting up linear c_adm embedding layer") + self.label_emb = nn.Linear(1, time_embed_dim) + else: + raise ValueError() + + self.input_blocks = nn.ModuleList( + [ + TimestepEmbedSequential( + conv_nd(dims, in_channels, model_channels, 3, padding=1) + ) + ] + ) + self._feature_size = model_channels + input_block_chans = [model_channels] + ch = model_channels + ds = 1 + for level, mult in enumerate(channel_mult): + for nr in range(self.num_res_blocks[level]): + layers = [ + ResBlock( + ch, + time_embed_dim, + dropout, + out_channels=mult * model_channels, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + ) + ] + ch = mult * model_channels + if ds in attention_resolutions: + if num_head_channels == -1: + dim_head = ch // num_heads + else: + num_heads = ch // num_head_channels + dim_head = num_head_channels + if legacy: + #num_heads = 1 + dim_head = ch // num_heads if use_spatial_transformer else num_head_channels + if exists(disable_self_attentions): + disabled_sa = disable_self_attentions[level] + else: + disabled_sa = False + + if not exists(num_attention_blocks) or nr < num_attention_blocks[level]: + layers.append( + AttentionBlock( + ch, + use_checkpoint=use_checkpoint, + num_heads=num_heads, + num_head_channels=dim_head, + use_new_attention_order=use_new_attention_order, + ) if not use_spatial_transformer else SpatialTransformer( + ch, num_heads, dim_head, depth=transformer_depth, context_dim=context_dim, + disable_self_attn=disabled_sa, use_linear=use_linear_in_transformer, + use_checkpoint=use_checkpoint + ) + ) + self.input_blocks.append(TimestepEmbedSequential(*layers)) + self._feature_size += ch + input_block_chans.append(ch) + if level != len(channel_mult) - 1: + out_ch = ch + self.input_blocks.append( + TimestepEmbedSequential( + ResBlock( + ch, + time_embed_dim, + dropout, + out_channels=out_ch, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + down=True, + ) + if resblock_updown + else Downsample( + ch, conv_resample, dims=dims, out_channels=out_ch + ) + ) + ) + ch = out_ch + input_block_chans.append(ch) + ds *= 2 + self._feature_size += ch + + if num_head_channels == -1: + dim_head = ch // num_heads + else: + num_heads = ch // num_head_channels + dim_head = num_head_channels + if legacy: + #num_heads = 1 + dim_head = ch // num_heads if use_spatial_transformer else num_head_channels + self.middle_block = TimestepEmbedSequential( + ResBlock( + ch, + time_embed_dim, + dropout, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + ), + AttentionBlock( + ch, + use_checkpoint=use_checkpoint, + num_heads=num_heads, + num_head_channels=dim_head, + use_new_attention_order=use_new_attention_order, + ) if not use_spatial_transformer else SpatialTransformer( # always uses a self-attn + ch, num_heads, dim_head, depth=transformer_depth, context_dim=context_dim, + disable_self_attn=disable_middle_self_attn, use_linear=use_linear_in_transformer, + use_checkpoint=use_checkpoint + ), + ResBlock( + ch, + time_embed_dim, + dropout, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + ), + ) + self._feature_size += ch + + self.output_blocks = nn.ModuleList([]) + for level, mult in list(enumerate(channel_mult))[::-1]: + for i in range(self.num_res_blocks[level] + 1): + ich = input_block_chans.pop() + layers = [ + ResBlock( + ch + ich, + time_embed_dim, + dropout, + out_channels=model_channels * mult, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + ) + ] + ch = model_channels * mult + if ds in attention_resolutions: + if num_head_channels == -1: + dim_head = ch // num_heads + else: + num_heads = ch // num_head_channels + dim_head = num_head_channels + if legacy: + #num_heads = 1 + dim_head = ch // num_heads if use_spatial_transformer else num_head_channels + if exists(disable_self_attentions): + disabled_sa = disable_self_attentions[level] + else: + disabled_sa = False + + if not exists(num_attention_blocks) or i < num_attention_blocks[level]: + layers.append( + AttentionBlock( + ch, + use_checkpoint=use_checkpoint, + num_heads=num_heads_upsample, + num_head_channels=dim_head, + use_new_attention_order=use_new_attention_order, + ) if not use_spatial_transformer else SpatialTransformer( + ch, num_heads, dim_head, depth=transformer_depth, context_dim=context_dim, + disable_self_attn=disabled_sa, use_linear=use_linear_in_transformer, + use_checkpoint=use_checkpoint + ) + ) + if level and i == self.num_res_blocks[level]: + out_ch = ch + layers.append( + ResBlock( + ch, + time_embed_dim, + dropout, + out_channels=out_ch, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + up=True, + ) + if resblock_updown + else Upsample(ch, conv_resample, dims=dims, out_channels=out_ch) + ) + ds //= 2 + self.output_blocks.append(TimestepEmbedSequential(*layers)) + self._feature_size += ch + + self.out = nn.Sequential( + normalization(ch), + nn.SiLU(), + zero_module(conv_nd(dims, model_channels, out_channels, 3, padding=1)), + ) + if self.predict_codebook_ids: + self.id_predictor = nn.Sequential( + normalization(ch), + conv_nd(dims, model_channels, n_embed, 1), + #nn.LogSoftmax(dim=1) # change to cross_entropy and produce non-normalized logits + ) + + def convert_to_fp16(self): + """ + Convert the torso of the model to float16. + """ + self.input_blocks.apply(convert_module_to_f16) + self.middle_block.apply(convert_module_to_f16) + self.output_blocks.apply(convert_module_to_f16) + + def convert_to_fp32(self): + """ + Convert the torso of the model to float32. + """ + self.input_blocks.apply(convert_module_to_f32) + self.middle_block.apply(convert_module_to_f32) + self.output_blocks.apply(convert_module_to_f32) + + def forward(self, x, timesteps=None, context=None, y=None,**kwargs): + """ + Apply the model to an input batch. + :param x: an [N x C x ...] Tensor of inputs. + :param timesteps: a 1-D batch of timesteps. + :param context: conditioning plugged in via crossattn + :param y: an [N] Tensor of labels, if class-conditional. + :return: an [N x C x ...] Tensor of outputs. + """ + # breakpoint() + assert (y is not None) == ( + self.num_classes is not None + ), "must specify y if and only if the model is class-conditional" + hs = [] + t_emb = timestep_embedding(timesteps, self.model_channels, repeat_only=False) + emb = self.time_embed(t_emb) + + if self.num_classes is not None: + assert y.shape[0] == x.shape[0] + emb = emb + self.label_emb(y) + + h = x.type(self.dtype) + # breakpoint() + for module in self.input_blocks: + h = module(h, emb, context) + hs.append(h) + h = self.middle_block(h, emb, context) + for module in self.output_blocks: + h = th.cat([h, hs.pop()], dim=1) + h = module(h, emb, context) + h = h.type(x.dtype) + if self.predict_codebook_ids: + return self.id_predictor(h) + else: + return self.out(h) diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/openaimodel_pseudo3D.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/openaimodel_pseudo3D.py new file mode 100644 index 0000000000000000000000000000000000000000..7fb8d9a01ce36ca7564e7dc1abbaec8d889ed5bc --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/openaimodel_pseudo3D.py @@ -0,0 +1,820 @@ +from abc import abstractmethod +from functools import partial +import math +from typing import Iterable + +from einops import rearrange +import numpy as np +import torch as th +import torch.nn as nn +import torch.nn.functional as F + +from ldm.modules.diffusionmodules.util import ( + checkpoint, + conv_nd, + linear, + avg_pool_nd, + zero_module, + normalization, + timestep_embedding, +) +from ldm.modules.attention import SpatialTransformer + + +def exists(val): + return val is not None + +def default(val, d): + return val if exists(val) else d + +# dummy replace +def convert_module_to_f16(x): + pass + +def convert_module_to_f32(x): + pass + +class PseudoConv3d(nn.Module): + def __init__( + self, + dim, + dim_out = None, + kernel_size = 3, + *, + temporal_kernel_size = None, + **kwargs + ): + super().__init__() + dim_out = default(dim_out, dim) + temporal_kernel_size = default(temporal_kernel_size, kernel_size) + + # self.temporal_conv = nn.Conv1d(dim_out, dim_out, kernel_size = temporal_kernel_size, padding = temporal_kernel_size // 2) if kernel_size > 1 else None + self.temporal_conv = nn.Conv1d(dim_out, dim_out, kernel_size = temporal_kernel_size, padding = temporal_kernel_size // 2, padding_mode='reflect') if kernel_size > 1 else None + + + if exists(self.temporal_conv): + nn.init.dirac_(self.temporal_conv.weight.data) # initialized to be identity + nn.init.zeros_(self.temporal_conv.bias.data) + + def forward( + self, + x, + enable_time = True + ): + _, _, h, w = x.shape + + if not enable_time or not exists(self.temporal_conv): + return x + + x = rearrange(x, '(b f) c h w -> (b h w) c f', f = 17) + x = self.temporal_conv(x) + x = rearrange(x, '(b h w) c f -> (b f) c h w', h = h, w = w) + + return x + + +## go +class AttentionPool2d(nn.Module): + """ + Adapted from CLIP: https://github.com/openai/CLIP/blob/main/clip/model.py + """ + + def __init__( + self, + spacial_dim: int, + embed_dim: int, + num_heads_channels: int, + output_dim: int = None, + ): + super().__init__() + self.positional_embedding = nn.Parameter(th.randn(embed_dim, spacial_dim ** 2 + 1) / embed_dim ** 0.5) + self.qkv_proj = conv_nd(1, embed_dim, 3 * embed_dim, 1) + self.c_proj = conv_nd(1, embed_dim, output_dim or embed_dim, 1) + self.num_heads = embed_dim // num_heads_channels + self.attention = QKVAttention(self.num_heads) + + def forward(self, x): + b, c, *_spatial = x.shape + x = x.reshape(b, c, -1) # NC(HW) + x = th.cat([x.mean(dim=-1, keepdim=True), x], dim=-1) # NC(HW+1) + x = x + self.positional_embedding[None, :, :].to(x.dtype) # NC(HW+1) + x = self.qkv_proj(x) + x = self.attention(x) + x = self.c_proj(x) + return x[:, :, 0] + + +class TimestepBlock(nn.Module): + """ + Any module where forward() takes timestep embeddings as a second argument. + """ + + @abstractmethod + def forward(self, x, emb): + """ + Apply the module to `x` given `emb` timestep embeddings. + """ + + +class TimestepEmbedSequential(nn.Sequential, TimestepBlock): + """ + A sequential module that passes timestep embeddings to the children that + support it as an extra input. + """ + + def forward(self, x, emb, context=None): + for layer in self: + if isinstance(layer, TimestepBlock): + x = layer(x, emb) + elif isinstance(layer, SpatialTransformer): + x = layer(x, context) + else: + x = layer(x) + return x + + +class Upsample(nn.Module): + """ + An upsampling layer with an optional convolution. + :param channels: channels in the inputs and outputs. + :param use_conv: a bool determining if a convolution is applied. + :param dims: determines if the signal is 1D, 2D, or 3D. If 3D, then + upsampling occurs in the inner-two dimensions. + """ + + def __init__(self, channels, use_conv, dims=2, out_channels=None, padding=1): + super().__init__() + self.channels = channels + self.out_channels = out_channels or channels + self.use_conv = use_conv + self.dims = dims + if use_conv: + self.conv = conv_nd(dims, self.channels, self.out_channels, 3, padding=padding) + self.conv_tem = PseudoConv3d(self.out_channels, self.out_channels, 3) + + def forward(self, x): + assert x.shape[1] == self.channels + if self.dims == 3: + x = F.interpolate( + x, (x.shape[2], x.shape[3] * 2, x.shape[4] * 2), mode="nearest" + ) + else: + x = F.interpolate(x, scale_factor=2, mode="nearest") + if self.use_conv: + x = self.conv(x) + x = self.conv_tem(x) + return x + +class TransposedUpsample(nn.Module): + 'Learned 2x upsampling without padding' + def __init__(self, channels, out_channels=None, ks=5): + super().__init__() + self.channels = channels + self.out_channels = out_channels or channels + + self.up = nn.ConvTranspose2d(self.channels,self.out_channels,kernel_size=ks,stride=2) + + def forward(self,x): + return self.up(x) + + +class Downsample(nn.Module): + """ + A downsampling layer with an optional convolution. + :param channels: channels in the inputs and outputs. + :param use_conv: a bool determining if a convolution is applied. + :param dims: determines if the signal is 1D, 2D, or 3D. If 3D, then + downsampling occurs in the inner-two dimensions. + """ + + def __init__(self, channels, use_conv, dims=2, out_channels=None,padding=1): + super().__init__() + self.channels = channels + self.out_channels = out_channels or channels + self.use_conv = use_conv + self.dims = dims + stride = 2 if dims != 3 else (1, 2, 2) + if use_conv: + self.op = conv_nd( + dims, self.channels, self.out_channels, 3, stride=stride, padding=padding + ) + self.op_tem = PseudoConv3d(self.out_channels, self.out_channels, 3) + else: + assert self.channels == self.out_channels + self.op = avg_pool_nd(dims, kernel_size=stride, stride=stride) + + def forward(self, x): + assert x.shape[1] == self.channels + if self.use_conv: + x = self.op(x) + x = self.op_tem(x) + else: + x = self.op(x) + return x + + +class ResBlock(TimestepBlock): + """ + A residual block that can optionally change the number of channels. + :param channels: the number of input channels. + :param emb_channels: the number of timestep embedding channels. + :param dropout: the rate of dropout. + :param out_channels: if specified, the number of out channels. + :param use_conv: if True and out_channels is specified, use a spatial + convolution instead of a smaller 1x1 convolution to change the + channels in the skip connection. + :param dims: determines if the signal is 1D, 2D, or 3D. + :param use_checkpoint: if True, use gradient checkpointing on this module. + :param up: if True, use this block for upsampling. + :param down: if True, use this block for downsampling. + """ + + def __init__( + self, + channels, + emb_channels, + dropout, + out_channels=None, + use_conv=False, + use_scale_shift_norm=False, + dims=2, + use_checkpoint=False, + up=False, + down=False, + ): + super().__init__() + self.channels = channels + self.emb_channels = emb_channels + self.dropout = dropout + self.out_channels = out_channels or channels + self.use_conv = use_conv + self.use_checkpoint = use_checkpoint + self.use_scale_shift_norm = use_scale_shift_norm + + self.in_layers = nn.Sequential( + normalization(channels), + nn.SiLU(), + conv_nd(dims, channels, self.out_channels, 3, padding=1), + ) + self.in_layers_tem = PseudoConv3d(channels, self.out_channels, 3) + + self.updown = up or down + + if up: + self.h_upd = Upsample(channels, False, dims) + self.x_upd = Upsample(channels, False, dims) + elif down: + self.h_upd = Downsample(channels, False, dims) + self.x_upd = Downsample(channels, False, dims) + else: + self.h_upd = self.x_upd = nn.Identity() + + self.emb_layers = nn.Sequential( + nn.SiLU(), + linear( + emb_channels, + 2 * self.out_channels if use_scale_shift_norm else self.out_channels, + ), + ) + self.out_layers = nn.Sequential( + normalization(self.out_channels), + nn.SiLU(), + nn.Dropout(p=dropout), + zero_module( + conv_nd(dims, self.out_channels, self.out_channels, 3, padding=1) + ), + ) + self.out_layers_tem = PseudoConv3d(self.out_channels, self.out_channels, 3) + + if self.out_channels == channels: + self.skip_connection = nn.Identity() + elif use_conv: + self.skip_connection = conv_nd( + dims, channels, self.out_channels, 3, padding=1 + ) + else: + self.skip_connection = conv_nd(dims, channels, self.out_channels, 1) + + def forward(self, x, emb): + """ + Apply the block to a Tensor, conditioned on a timestep embedding. + :param x: an [N x C x ...] Tensor of features. + :param emb: an [N x emb_channels] Tensor of timestep embeddings. + :return: an [N x C x ...] Tensor of outputs. + """ + return checkpoint( + self._forward, (x, emb), self.parameters(), self.use_checkpoint + ) + + + def _forward(self, x, emb): + if self.updown: + in_rest, in_conv = self.in_layers[:-1], self.in_layers[-1] + h = in_rest(x) + h = self.h_upd(h) + x = self.x_upd(x) + h = in_conv(h) + h = self.in_layers_tem(h) + else: + h = self.in_layers(x) + h = self.in_layers_tem(h) + emb_out = self.emb_layers(emb).type(h.dtype) + while len(emb_out.shape) < len(h.shape): + emb_out = emb_out[..., None] + if self.use_scale_shift_norm: + out_norm, out_rest = self.out_layers[0], self.out_layers[1:] + scale, shift = th.chunk(emb_out, 2, dim=1) + h = out_norm(h) * (1 + scale) + shift + h = out_rest(h) + h = self.out_layers_tem(h) + else: + h = h + emb_out + h = self.out_layers(h) + h = self.out_layers_tem(h) + return self.skip_connection(x) + h + + +class AttentionBlock(nn.Module): + """ + An attention block that allows spatial positions to attend to each other. + Originally ported from here, but adapted to the N-d case. + https://github.com/hojonathanho/diffusion/blob/1e0dceb3b3495bbe19116a5e1b3596cd0706c543/diffusion_tf/models/unet.py#L66. + """ + + def __init__( + self, + channels, + num_heads=1, + num_head_channels=-1, + use_checkpoint=False, + use_new_attention_order=False, + ): + super().__init__() + self.channels = channels + if num_head_channels == -1: + self.num_heads = num_heads + else: + assert ( + channels % num_head_channels == 0 + ), f"q,k,v channels {channels} is not divisible by num_head_channels {num_head_channels}" + self.num_heads = channels // num_head_channels + self.use_checkpoint = use_checkpoint + self.norm = normalization(channels) + self.qkv = conv_nd(1, channels, channels * 3, 1) + self.qkv_tem = conv_nd(1, channels, channels * 3, 1) + + if use_new_attention_order: + # split qkv before split heads + self.attention = QKVAttention(self.num_heads) + self.attention_tem = QKVAttention(self.num_heads) + else: + # split heads before split qkv + self.attention = QKVAttentionLegacy(self.num_heads) + self.attention_tem = QKVAttentionLegacy(self.num_heads) + + self.proj_out = zero_module(conv_nd(1, channels, channels, 1)) + self.proj_out_tem = zero_module(conv_nd(1, channels, channels, 1)) + + def forward(self, x): + return checkpoint(self._forward, (x,), self.parameters(), True) # TODO: check checkpoint usage, is True # TODO: fix the .half call!!! + #return pt_checkpoint(self._forward, x) # pytorch + + def _forward(self, x): + b, c, *spatial = x.shape + x = x.reshape(b, c, -1) + qkv = self.qkv(self.norm(x)) + h = self.attention(qkv) + h = self.proj_out(h) + + x = (x + h).reshape(b, c, *spatial) + x = rearrange(x, '(b f) c h w -> (b h w) c f', f = 16) + + qkv_tem = self.qkv_tem(self.norm(x)) + h_tem = self.attention_tem(qkv_tem) + h_tem = self.proj_out_tem(h_tem) + + x = x+h_tem + x = rearrange(x, '(b h w) c f -> (b f) c h w', w = spatial[0], h = spatial[1]) + return x + + +def count_flops_attn(model, _x, y): + """ + A counter for the `thop` package to count the operations in an + attention operation. + Meant to be used like: + macs, params = thop.profile( + model, + inputs=(inputs, timestamps), + custom_ops={QKVAttention: QKVAttention.count_flops}, + ) + """ + b, c, *spatial = y[0].shape + num_spatial = int(np.prod(spatial)) + # We perform two matmuls with the same number of ops. + # The first computes the weight matrix, the second computes + # the combination of the value vectors. + matmul_ops = 2 * b * (num_spatial ** 2) * c + model.total_ops += th.DoubleTensor([matmul_ops]) + + +class QKVAttentionLegacy(nn.Module): + """ + A module which performs QKV attention. Matches legacy QKVAttention + input/ouput heads shaping + """ + + def __init__(self, n_heads): + super().__init__() + self.n_heads = n_heads + + def forward(self, qkv): + """ + Apply QKV attention. + :param qkv: an [N x (H * 3 * C) x T] tensor of Qs, Ks, and Vs. + :return: an [N x (H * C) x T] tensor after attention. + """ + bs, width, length = qkv.shape + assert width % (3 * self.n_heads) == 0 + ch = width // (3 * self.n_heads) + q, k, v = qkv.reshape(bs * self.n_heads, ch * 3, length).split(ch, dim=1) + scale = 1 / math.sqrt(math.sqrt(ch)) + weight = th.einsum( + "bct,bcs->bts", q * scale, k * scale + ) # More stable with f16 than dividing afterwards + weight = th.softmax(weight.float(), dim=-1).type(weight.dtype) + a = th.einsum("bts,bcs->bct", weight, v) + return a.reshape(bs, -1, length) + + @staticmethod + def count_flops(model, _x, y): + return count_flops_attn(model, _x, y) + + +class QKVAttention(nn.Module): + """ + A module which performs QKV attention and splits in a different order. + """ + + def __init__(self, n_heads): + super().__init__() + self.n_heads = n_heads + + def forward(self, qkv): + """ + Apply QKV attention. + :param qkv: an [N x (3 * H * C) x T] tensor of Qs, Ks, and Vs. + :return: an [N x (H * C) x T] tensor after attention. + """ + bs, width, length = qkv.shape + assert width % (3 * self.n_heads) == 0 + ch = width // (3 * self.n_heads) + q, k, v = qkv.chunk(3, dim=1) + scale = 1 / math.sqrt(math.sqrt(ch)) + weight = th.einsum( + "bct,bcs->bts", + (q * scale).view(bs * self.n_heads, ch, length), + (k * scale).view(bs * self.n_heads, ch, length), + ) # More stable with f16 than dividing afterwards + weight = th.softmax(weight.float(), dim=-1).type(weight.dtype) + a = th.einsum("bts,bcs->bct", weight, v.reshape(bs * self.n_heads, ch, length)) + return a.reshape(bs, -1, length) + + @staticmethod + def count_flops(model, _x, y): + return count_flops_attn(model, _x, y) + + +class UNetModel(nn.Module): + """ + The full UNet model with attention and timestep embedding. + :param in_channels: channels in the input Tensor. + :param model_channels: base channel count for the model. + :param out_channels: channels in the output Tensor. + :param num_res_blocks: number of residual blocks per downsample. + :param attention_resolutions: a collection of downsample rates at which + attention will take place. May be a set, list, or tuple. + For example, if this contains 4, then at 4x downsampling, attention + will be used. + :param dropout: the dropout probability. + :param channel_mult: channel multiplier for each level of the UNet. + :param conv_resample: if True, use learned convolutions for upsampling and + downsampling. + :param dims: determines if the signal is 1D, 2D, or 3D. + :param num_classes: if specified (as an int), then this model will be + class-conditional with `num_classes` classes. + :param use_checkpoint: use gradient checkpointing to reduce memory usage. + :param num_heads: the number of attention heads in each attention layer. + :param num_heads_channels: if specified, ignore num_heads and instead use + a fixed channel width per attention head. + :param num_heads_upsample: works with num_heads to set a different number + of heads for upsampling. Deprecated. + :param use_scale_shift_norm: use a FiLM-like conditioning mechanism. + :param resblock_updown: use residual blocks for up/downsampling. + :param use_new_attention_order: use a different attention pattern for potentially + increased efficiency. + """ + + def __init__( + self, + image_size, + in_channels, + model_channels, + out_channels, + num_res_blocks, + attention_resolutions, + dropout=0, + channel_mult=(1, 2, 4, 8), + conv_resample=True, + dims=2, + num_classes=None, + use_checkpoint=False, + use_fp16=False, + num_heads=-1, + num_head_channels=-1, + num_heads_upsample=-1, + use_scale_shift_norm=False, + resblock_updown=False, + use_new_attention_order=False, + use_spatial_transformer=False, # custom transformer support + transformer_depth=1, # custom transformer support + context_dim=None, # custom transformer support + n_embed=None, # custom support for prediction of discrete ids into codebook of first stage vq model + legacy=True, + ): + super().__init__() + if use_spatial_transformer: + assert context_dim is not None, 'Fool!! You forgot to include the dimension of your cross-attention conditioning...' + + if context_dim is not None: + assert use_spatial_transformer, 'Fool!! You forgot to use the spatial transformer for your cross-attention conditioning...' + from omegaconf.listconfig import ListConfig + if type(context_dim) == ListConfig: + context_dim = list(context_dim) + + if num_heads_upsample == -1: + num_heads_upsample = num_heads + + if num_heads == -1: + assert num_head_channels != -1, 'Either num_heads or num_head_channels has to be set' + + if num_head_channels == -1: + assert num_heads != -1, 'Either num_heads or num_head_channels has to be set' + + self.image_size = image_size + self.in_channels = in_channels + self.model_channels = model_channels + self.out_channels = out_channels + self.num_res_blocks = num_res_blocks + self.attention_resolutions = attention_resolutions + self.dropout = dropout + self.channel_mult = channel_mult + self.conv_resample = conv_resample + self.num_classes = num_classes + self.use_checkpoint = use_checkpoint + self.dtype = th.float16 if use_fp16 else th.float32 + self.num_heads = num_heads + self.num_head_channels = num_head_channels + self.num_heads_upsample = num_heads_upsample + self.predict_codebook_ids = n_embed is not None + + time_embed_dim = model_channels * 4 + self.time_embed = nn.Sequential( + linear(model_channels, time_embed_dim), + nn.SiLU(), + linear(time_embed_dim, time_embed_dim), + ) + + if self.num_classes is not None: + self.label_emb = nn.Embedding(num_classes, time_embed_dim) + + self.input_blocks = nn.ModuleList( + [ + TimestepEmbedSequential( + conv_nd(dims, in_channels, model_channels, 3, padding=1), + PseudoConv3d(model_channels, model_channels, 3) + ) + ] + ) + + self._feature_size = model_channels + input_block_chans = [model_channels] + ch = model_channels + ds = 1 + for level, mult in enumerate(channel_mult): + for _ in range(num_res_blocks): + layers = [ + ResBlock( + ch, + time_embed_dim, + dropout, + out_channels=mult * model_channels, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + ) + ] + ch = mult * model_channels + if ds in attention_resolutions: + if num_head_channels == -1: + dim_head = ch // num_heads + else: + num_heads = ch // num_head_channels + dim_head = num_head_channels + if legacy: + #num_heads = 1 + dim_head = ch // num_heads if use_spatial_transformer else num_head_channels + layers.append( + AttentionBlock( + ch, + use_checkpoint=use_checkpoint, + num_heads=num_heads, + num_head_channels=dim_head, + use_new_attention_order=use_new_attention_order, + ) if not use_spatial_transformer else SpatialTransformer( + ch, num_heads, dim_head, depth=transformer_depth, context_dim=context_dim, use_checkpoint=use_checkpoint + ) + ) + self.input_blocks.append(TimestepEmbedSequential(*layers)) + self._feature_size += ch + input_block_chans.append(ch) + if level != len(channel_mult) - 1: + out_ch = ch + self.input_blocks.append( + TimestepEmbedSequential( + ResBlock( + ch, + time_embed_dim, + dropout, + out_channels=out_ch, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + down=True, + ) + if resblock_updown + else Downsample( + ch, conv_resample, dims=dims, out_channels=out_ch + ) + ) + ) + ch = out_ch + input_block_chans.append(ch) + ds *= 2 + self._feature_size += ch + + if num_head_channels == -1: + dim_head = ch // num_heads + else: + num_heads = ch // num_head_channels + dim_head = num_head_channels + if legacy: + #num_heads = 1 + dim_head = ch // num_heads if use_spatial_transformer else num_head_channels + self.middle_block = TimestepEmbedSequential( + ResBlock( + ch, + time_embed_dim, + dropout, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + ), + AttentionBlock( + ch, + use_checkpoint=use_checkpoint, + num_heads=num_heads, + num_head_channels=dim_head, + use_new_attention_order=use_new_attention_order, + ) if not use_spatial_transformer else SpatialTransformer( + ch, num_heads, dim_head, depth=transformer_depth, context_dim=context_dim, use_checkpoint=use_checkpoint + ), + ResBlock( + ch, + time_embed_dim, + dropout, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + ), + ) + self._feature_size += ch + + self.output_blocks = nn.ModuleList([]) + for level, mult in list(enumerate(channel_mult))[::-1]: + for i in range(num_res_blocks + 1): + ich = input_block_chans.pop() + layers = [ + ResBlock( + ch + ich, + time_embed_dim, + dropout, + out_channels=model_channels * mult, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + ) + ] + ch = model_channels * mult + if ds in attention_resolutions: + if num_head_channels == -1: + dim_head = ch // num_heads + else: + num_heads = ch // num_head_channels + dim_head = num_head_channels + if legacy: + #num_heads = 1 + dim_head = ch // num_heads if use_spatial_transformer else num_head_channels + layers.append( + AttentionBlock( + ch, + use_checkpoint=use_checkpoint, + num_heads=num_heads_upsample, + num_head_channels=dim_head, + use_new_attention_order=use_new_attention_order, + ) if not use_spatial_transformer else SpatialTransformer( + ch, num_heads, dim_head, depth=transformer_depth, context_dim=context_dim, use_checkpoint=use_checkpoint + ) + ) + if level and i == num_res_blocks: + out_ch = ch + layers.append( + ResBlock( + ch, + time_embed_dim, + dropout, + out_channels=out_ch, + dims=dims, + use_checkpoint=use_checkpoint, + use_scale_shift_norm=use_scale_shift_norm, + up=True, + ) + if resblock_updown + else Upsample(ch, conv_resample, dims=dims, out_channels=out_ch) + ) + ds //= 2 + self.output_blocks.append(TimestepEmbedSequential(*layers)) + self._feature_size += ch + + self.out = nn.Sequential( + normalization(ch), + nn.SiLU(), + zero_module(conv_nd(dims, model_channels, out_channels, 3, padding=1)), + ) + self.out_tem = PseudoConv3d(out_channels, out_channels, 3) + + if self.predict_codebook_ids: + self.id_predictor = nn.Sequential( + normalization(ch), + conv_nd(dims, model_channels, n_embed, 1), + #nn.LogSoftmax(dim=1) # change to cross_entropy and produce non-normalized logits + ) + + def convert_to_fp16(self): + """ + Convert the torso of the model to float16. + """ + self.input_blocks.apply(convert_module_to_f16) + self.middle_block.apply(convert_module_to_f16) + self.output_blocks.apply(convert_module_to_f16) + + def convert_to_fp32(self): + """ + Convert the torso of the model to float32. + """ + self.input_blocks.apply(convert_module_to_f32) + self.middle_block.apply(convert_module_to_f32) + self.output_blocks.apply(convert_module_to_f32) + + def forward(self, x, timesteps=None, context=None, y=None,**kwargs): + """ + Apply the model to an input batch. + :param x: an [N x C x ...] Tensor of inputs. + :param timesteps: a 1-D batch of timesteps. + :param context: conditioning plugged in via crossattn + :param y: an [N] Tensor of labels, if class-conditional. + :return: an [N x C x ...] Tensor of outputs. + """ + assert (y is not None) == ( + self.num_classes is not None + ), "must specify y if and only if the model is class-conditional" + hs = [] + t_emb = timestep_embedding(timesteps, self.model_channels, repeat_only=False) + emb = self.time_embed(t_emb) + + if self.num_classes is not None: + assert y.shape == (x.shape[0],) + emb = emb + self.label_emb(y) + + h = x.type(self.dtype) + for module in self.input_blocks: + h = module(h, emb, context) + hs.append(h) + h = self.middle_block(h, emb, context) + for module in self.output_blocks: + h = th.cat([h, hs.pop()], dim=1) + h = module(h, emb, context) + h = h.type(x.dtype) + if self.predict_codebook_ids: + return self.id_predictor(h) + else: + return self.out_tem(self.out(h)) \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/upscaling.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/upscaling.py new file mode 100644 index 0000000000000000000000000000000000000000..03816662098ce1ffac79bd939b892e867ab91988 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/upscaling.py @@ -0,0 +1,81 @@ +import torch +import torch.nn as nn +import numpy as np +from functools import partial + +from ldm.modules.diffusionmodules.util import extract_into_tensor, make_beta_schedule +from ldm.util import default + + +class AbstractLowScaleModel(nn.Module): + # for concatenating a downsampled image to the latent representation + def __init__(self, noise_schedule_config=None): + super(AbstractLowScaleModel, self).__init__() + if noise_schedule_config is not None: + self.register_schedule(**noise_schedule_config) + + def register_schedule(self, beta_schedule="linear", timesteps=1000, + linear_start=1e-4, linear_end=2e-2, cosine_s=8e-3): + betas = make_beta_schedule(beta_schedule, timesteps, linear_start=linear_start, linear_end=linear_end, + cosine_s=cosine_s) + alphas = 1. - betas + alphas_cumprod = np.cumprod(alphas, axis=0) + alphas_cumprod_prev = np.append(1., alphas_cumprod[:-1]) + + timesteps, = betas.shape + self.num_timesteps = int(timesteps) + self.linear_start = linear_start + self.linear_end = linear_end + assert alphas_cumprod.shape[0] == self.num_timesteps, 'alphas have to be defined for each timestep' + + to_torch = partial(torch.tensor, dtype=torch.float32) + + self.register_buffer('betas', to_torch(betas)) + self.register_buffer('alphas_cumprod', to_torch(alphas_cumprod)) + self.register_buffer('alphas_cumprod_prev', to_torch(alphas_cumprod_prev)) + + # calculations for diffusion q(x_t | x_{t-1}) and others + self.register_buffer('sqrt_alphas_cumprod', to_torch(np.sqrt(alphas_cumprod))) + self.register_buffer('sqrt_one_minus_alphas_cumprod', to_torch(np.sqrt(1. - alphas_cumprod))) + self.register_buffer('log_one_minus_alphas_cumprod', to_torch(np.log(1. - alphas_cumprod))) + self.register_buffer('sqrt_recip_alphas_cumprod', to_torch(np.sqrt(1. / alphas_cumprod))) + self.register_buffer('sqrt_recipm1_alphas_cumprod', to_torch(np.sqrt(1. / alphas_cumprod - 1))) + + def q_sample(self, x_start, t, noise=None): + noise = default(noise, lambda: torch.randn_like(x_start)) + return (extract_into_tensor(self.sqrt_alphas_cumprod, t, x_start.shape) * x_start + + extract_into_tensor(self.sqrt_one_minus_alphas_cumprod, t, x_start.shape) * noise) + + def forward(self, x): + return x, None + + def decode(self, x): + return x + + +class SimpleImageConcat(AbstractLowScaleModel): + # no noise level conditioning + def __init__(self): + super(SimpleImageConcat, self).__init__(noise_schedule_config=None) + self.max_noise_level = 0 + + def forward(self, x): + # fix to constant noise level + return x, torch.zeros(x.shape[0], device=x.device).long() + + +class ImageConcatWithNoiseAugmentation(AbstractLowScaleModel): + def __init__(self, noise_schedule_config, max_noise_level=1000, to_cuda=False): + super().__init__(noise_schedule_config=noise_schedule_config) + self.max_noise_level = max_noise_level + + def forward(self, x, noise_level=None): + if noise_level is None: + noise_level = torch.randint(0, self.max_noise_level, (x.shape[0],), device=x.device).long() + else: + assert isinstance(noise_level, torch.Tensor) + z = self.q_sample(x, noise_level) + return z, noise_level + + + diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/util.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/util.py new file mode 100644 index 0000000000000000000000000000000000000000..637363dfe34799e70cfdbcd11445212df9d9ca1f --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/modules/diffusionmodules/util.py @@ -0,0 +1,270 @@ +# adopted from +# https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/gaussian_diffusion.py +# and +# https://github.com/lucidrains/denoising-diffusion-pytorch/blob/7706bdfc6f527f58d33f84b7b522e61e6e3164b3/denoising_diffusion_pytorch/denoising_diffusion_pytorch.py +# and +# https://github.com/openai/guided-diffusion/blob/0ba878e517b276c45d1195eb29f6f5f72659a05b/guided_diffusion/nn.py +# +# thanks! + + +import os +import math +import torch +import torch.nn as nn +import numpy as np +from einops import repeat + +from ldm.util import instantiate_from_config + + +def make_beta_schedule(schedule, n_timestep, linear_start=1e-4, linear_end=2e-2, cosine_s=8e-3): + if schedule == "linear": + betas = ( + torch.linspace(linear_start ** 0.5, linear_end ** 0.5, n_timestep, dtype=torch.float64) ** 2 + ) + + elif schedule == "cosine": + timesteps = ( + torch.arange(n_timestep + 1, dtype=torch.float64) / n_timestep + cosine_s + ) + alphas = timesteps / (1 + cosine_s) * np.pi / 2 + alphas = torch.cos(alphas).pow(2) + alphas = alphas / alphas[0] + betas = 1 - alphas[1:] / alphas[:-1] + betas = np.clip(betas, a_min=0, a_max=0.999) + + elif schedule == "sqrt_linear": + betas = torch.linspace(linear_start, linear_end, n_timestep, dtype=torch.float64) + elif schedule == "sqrt": + betas = torch.linspace(linear_start, linear_end, n_timestep, dtype=torch.float64) ** 0.5 + else: + raise ValueError(f"schedule '{schedule}' unknown.") + return betas.numpy() + + +def make_ddim_timesteps(ddim_discr_method, num_ddim_timesteps, num_ddpm_timesteps, verbose=True): + if ddim_discr_method == 'uniform': + c = num_ddpm_timesteps // num_ddim_timesteps + ddim_timesteps = np.asarray(list(range(0, num_ddpm_timesteps, c))) + elif ddim_discr_method == 'quad': + ddim_timesteps = ((np.linspace(0, np.sqrt(num_ddpm_timesteps * .8), num_ddim_timesteps)) ** 2).astype(int) + else: + raise NotImplementedError(f'There is no ddim discretization method called "{ddim_discr_method}"') + + # assert ddim_timesteps.shape[0] == num_ddim_timesteps + # add one to get the final alpha values right (the ones from first scale to data during sampling) + steps_out = ddim_timesteps + 1 + if verbose: + print(f'Selected timesteps for ddim sampler: {steps_out}') + return steps_out + + +def make_ddim_sampling_parameters(alphacums, ddim_timesteps, eta, verbose=True): + # select alphas for computing the variance schedule + alphas = alphacums[ddim_timesteps] + alphas_prev = np.asarray([alphacums[0]] + alphacums[ddim_timesteps[:-1]].tolist()) + + # according the the formula provided in https://arxiv.org/abs/2010.02502 + sigmas = eta * np.sqrt((1 - alphas_prev) / (1 - alphas) * (1 - alphas / alphas_prev)) + if verbose: + print(f'Selected alphas for ddim sampler: a_t: {alphas}; a_(t-1): {alphas_prev}') + print(f'For the chosen value of eta, which is {eta}, ' + f'this results in the following sigma_t schedule for ddim sampler {sigmas}') + return sigmas, alphas, alphas_prev + + +def betas_for_alpha_bar(num_diffusion_timesteps, alpha_bar, max_beta=0.999): + """ + Create a beta schedule that discretizes the given alpha_t_bar function, + which defines the cumulative product of (1-beta) over time from t = [0,1]. + :param num_diffusion_timesteps: the number of betas to produce. + :param alpha_bar: a lambda that takes an argument t from 0 to 1 and + produces the cumulative product of (1-beta) up to that + part of the diffusion process. + :param max_beta: the maximum beta to use; use values lower than 1 to + prevent singularities. + """ + betas = [] + for i in range(num_diffusion_timesteps): + t1 = i / num_diffusion_timesteps + t2 = (i + 1) / num_diffusion_timesteps + betas.append(min(1 - alpha_bar(t2) / alpha_bar(t1), max_beta)) + return np.array(betas) + + +def extract_into_tensor(a, t, x_shape): + b, *_ = t.shape + out = a.gather(-1, t) + return out.reshape(b, *((1,) * (len(x_shape) - 1))) + + +def checkpoint(func, inputs, params, flag): + """ + Evaluate a function without caching intermediate activations, allowing for + reduced memory at the expense of extra compute in the backward pass. + :param func: the function to evaluate. + :param inputs: the argument sequence to pass to `func`. + :param params: a sequence of parameters `func` depends on but does not + explicitly take as arguments. + :param flag: if False, disable gradient checkpointing. + """ + if flag: + args = tuple(inputs) + tuple(params) + return CheckpointFunction.apply(func, len(inputs), *args) + else: + return func(*inputs) + + +class CheckpointFunction(torch.autograd.Function): + @staticmethod + def forward(ctx, run_function, length, *args): + ctx.run_function = run_function + ctx.input_tensors = list(args[:length]) + ctx.input_params = list(args[length:]) + ctx.gpu_autocast_kwargs = {"enabled": torch.is_autocast_enabled(), + "dtype": torch.get_autocast_gpu_dtype(), + "cache_enabled": torch.is_autocast_cache_enabled()} + with torch.no_grad(): + output_tensors = ctx.run_function(*ctx.input_tensors) + return output_tensors + + @staticmethod + def backward(ctx, *output_grads): + ctx.input_tensors = [x.detach().requires_grad_(True) for x in ctx.input_tensors] + with torch.enable_grad(), \ + torch.cuda.amp.autocast(**ctx.gpu_autocast_kwargs): + # Fixes a bug where the first op in run_function modifies the + # Tensor storage in place, which is not allowed for detach()'d + # Tensors. + shallow_copies = [x.view_as(x) for x in ctx.input_tensors] + output_tensors = ctx.run_function(*shallow_copies) + input_grads = torch.autograd.grad( + output_tensors, + ctx.input_tensors + ctx.input_params, + output_grads, + allow_unused=True, + ) + del ctx.input_tensors + del ctx.input_params + del output_tensors + return (None, None) + input_grads + + +def timestep_embedding(timesteps, dim, max_period=10000, repeat_only=False): + """ + Create sinusoidal timestep embeddings. + :param timesteps: a 1-D Tensor of N indices, one per batch element. + These may be fractional. + :param dim: the dimension of the output. + :param max_period: controls the minimum frequency of the embeddings. + :return: an [N x dim] Tensor of positional embeddings. + """ + if not repeat_only: + half = dim // 2 + freqs = torch.exp( + -math.log(max_period) * torch.arange(start=0, end=half, dtype=torch.float32) / half + ).to(device=timesteps.device) + args = timesteps[:, None].float() * freqs[None] + embedding = torch.cat([torch.cos(args), torch.sin(args)], dim=-1) + if dim % 2: + embedding = torch.cat([embedding, torch.zeros_like(embedding[:, :1])], dim=-1) + else: + embedding = repeat(timesteps, 'b -> b d', d=dim) + return embedding + + +def zero_module(module): + """ + Zero out the parameters of a module and return it. + """ + for p in module.parameters(): + p.detach().zero_() + return module + + +def scale_module(module, scale): + """ + Scale the parameters of a module and return it. + """ + for p in module.parameters(): + p.detach().mul_(scale) + return module + + +def mean_flat(tensor): + """ + Take the mean over all non-batch dimensions. + """ + return tensor.mean(dim=list(range(1, len(tensor.shape)))) + + +def normalization(channels): + """ + Make a standard normalization layer. + :param channels: number of input channels. + :return: an nn.Module for normalization. + """ + return GroupNorm32(32, channels) + + +# PyTorch 1.7 has SiLU, but we support PyTorch 1.5. +class SiLU(nn.Module): + def forward(self, x): + return x * torch.sigmoid(x) + + +class GroupNorm32(nn.GroupNorm): + def forward(self, x): + return super().forward(x.float()).type(x.dtype) + +def conv_nd(dims, *args, **kwargs): + """ + Create a 1D, 2D, or 3D convolution module. + """ + if dims == 1: + return nn.Conv1d(*args, **kwargs) + elif dims == 2: + return nn.Conv2d(*args, **kwargs) + elif dims == 3: + return nn.Conv3d(*args, **kwargs) + raise ValueError(f"unsupported dimensions: {dims}") + + +def linear(*args, **kwargs): + """ + Create a linear module. + """ + return nn.Linear(*args, **kwargs) + + +def avg_pool_nd(dims, *args, **kwargs): + """ + Create a 1D, 2D, or 3D average pooling module. + """ + if dims == 1: + return nn.AvgPool1d(*args, **kwargs) + elif dims == 2: + return nn.AvgPool2d(*args, **kwargs) + elif dims == 3: + return nn.AvgPool3d(*args, **kwargs) + raise ValueError(f"unsupported dimensions: {dims}") + + +class HybridConditioner(nn.Module): + + def __init__(self, c_concat_config, c_crossattn_config): + super().__init__() + self.concat_conditioner = instantiate_from_config(c_concat_config) + self.crossattn_conditioner = instantiate_from_config(c_crossattn_config) + + def forward(self, c_concat, c_crossattn): + c_concat = self.concat_conditioner(c_concat) + c_crossattn = self.crossattn_conditioner(c_crossattn) + return {'c_concat': [c_concat], 'c_crossattn': [c_crossattn]} + + +def noise_like(shape, device, repeat=False): + repeat_noise = lambda: torch.randn((1, *shape[1:]), device=device).repeat(shape[0], *((1,) * (len(shape) - 1))) + noise = lambda: torch.randn(shape, device=device) + return repeat_noise() if repeat else noise() \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__init__.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..6244883baa2fb28c9b4d91ac4a7c7ae196da591e Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..cb43b48c4d257a70962712243307a2daf9b902cd Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..652d35e878b348b105b4de9eddeccf69d23c3bb1 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..f50ac65622d205ea2edac9e1ab4bc66a19551571 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/__init__.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..31bfe2ac2199459979d044262b7cf33ca3df2ac8 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..5ee0b1f11db5e83c50504ad956227d61d58545bb Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-38.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-38.pyc new file mode 100644 index 0000000000000000000000000000000000000000..4aaa94beadf284e8388b61bd73162c4bcea2fb45 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-38.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..4aeba3a7c397c569e3848671f9f529ba2ba54699 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/__pycache__/distributions.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/distributions.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/distributions.py new file mode 100644 index 0000000000000000000000000000000000000000..f2b8ef901130efc171aa69742ca0244d94d3f2e9 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/modules/distributions/distributions.py @@ -0,0 +1,92 @@ +import torch +import numpy as np + + +class AbstractDistribution: + def sample(self): + raise NotImplementedError() + + def mode(self): + raise NotImplementedError() + + +class DiracDistribution(AbstractDistribution): + def __init__(self, value): + self.value = value + + def sample(self): + return self.value + + def mode(self): + return self.value + + +class DiagonalGaussianDistribution(object): + def __init__(self, parameters, deterministic=False): + self.parameters = parameters + self.mean, self.logvar = torch.chunk(parameters, 2, dim=1) + self.logvar = torch.clamp(self.logvar, -30.0, 20.0) + self.deterministic = deterministic + self.std = torch.exp(0.5 * self.logvar) + self.var = torch.exp(self.logvar) + if self.deterministic: + self.var = self.std = torch.zeros_like(self.mean).to(device=self.parameters.device) + + def sample(self): + x = self.mean + self.std * torch.randn(self.mean.shape).to(device=self.parameters.device) + return x + + def kl(self, other=None): + if self.deterministic: + return torch.Tensor([0.]) + else: + if other is None: + return 0.5 * torch.sum(torch.pow(self.mean, 2) + + self.var - 1.0 - self.logvar, + dim=[1, 2, 3]) + else: + return 0.5 * torch.sum( + torch.pow(self.mean - other.mean, 2) / other.var + + self.var / other.var - 1.0 - self.logvar + other.logvar, + dim=[1, 2, 3]) + + def nll(self, sample, dims=[1,2,3]): + if self.deterministic: + return torch.Tensor([0.]) + logtwopi = np.log(2.0 * np.pi) + return 0.5 * torch.sum( + logtwopi + self.logvar + torch.pow(sample - self.mean, 2) / self.var, + dim=dims) + + def mode(self): + return self.mean + + +def normal_kl(mean1, logvar1, mean2, logvar2): + """ + source: https://github.com/openai/guided-diffusion/blob/27c20a8fab9cb472df5d6bdd6c8d11c8f430b924/guided_diffusion/losses.py#L12 + Compute the KL divergence between two gaussians. + Shapes are automatically broadcasted, so batches can be compared to + scalars, among other use cases. + """ + tensor = None + for obj in (mean1, logvar1, mean2, logvar2): + if isinstance(obj, torch.Tensor): + tensor = obj + break + assert tensor is not None, "at least one argument must be a Tensor" + + # Force variances to be Tensors. Broadcasting helps convert scalars to + # Tensors, but it does not work for torch.exp(). + logvar1, logvar2 = [ + x if isinstance(x, torch.Tensor) else torch.tensor(x).to(tensor) + for x in (logvar1, logvar2) + ] + + return 0.5 * ( + -1.0 + + logvar2 + - logvar1 + + torch.exp(logvar1 - logvar2) + + ((mean1 - mean2) ** 2) * torch.exp(-logvar2) + ) diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/ema.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/ema.py new file mode 100644 index 0000000000000000000000000000000000000000..bded25019b9bcbcd0260f0b8185f8c7859ca58c4 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/modules/ema.py @@ -0,0 +1,80 @@ +import torch +from torch import nn + + +class LitEma(nn.Module): + def __init__(self, model, decay=0.9999, use_num_upates=True): + super().__init__() + if decay < 0.0 or decay > 1.0: + raise ValueError('Decay must be between 0 and 1') + + self.m_name2s_name = {} + self.register_buffer('decay', torch.tensor(decay, dtype=torch.float32)) + self.register_buffer('num_updates', torch.tensor(0, dtype=torch.int) if use_num_upates + else torch.tensor(-1, dtype=torch.int)) + + for name, p in model.named_parameters(): + if p.requires_grad: + # remove as '.'-character is not allowed in buffers + s_name = name.replace('.', '') + self.m_name2s_name.update({name: s_name}) + self.register_buffer(s_name, p.clone().detach().data) + + self.collected_params = [] + + def reset_num_updates(self): + del self.num_updates + self.register_buffer('num_updates', torch.tensor(0, dtype=torch.int)) + + def forward(self, model): + decay = self.decay + + if self.num_updates >= 0: + self.num_updates += 1 + decay = min(self.decay, (1 + self.num_updates) / (10 + self.num_updates)) + + one_minus_decay = 1.0 - decay + + with torch.no_grad(): + m_param = dict(model.named_parameters()) + shadow_params = dict(self.named_buffers()) + + for key in m_param: + if m_param[key].requires_grad: + sname = self.m_name2s_name[key] + shadow_params[sname] = shadow_params[sname].type_as(m_param[key]) + shadow_params[sname].sub_(one_minus_decay * (shadow_params[sname] - m_param[key])) + else: + assert not key in self.m_name2s_name + + def copy_to(self, model): + m_param = dict(model.named_parameters()) + shadow_params = dict(self.named_buffers()) + for key in m_param: + if m_param[key].requires_grad: + m_param[key].data.copy_(shadow_params[self.m_name2s_name[key]].data) + else: + assert not key in self.m_name2s_name + + def store(self, parameters): + """ + Save the current parameters for restoring later. + Args: + parameters: Iterable of `torch.nn.Parameter`; the parameters to be + temporarily stored. + """ + self.collected_params = [param.clone() for param in parameters] + + def restore(self, parameters): + """ + Restore the parameters stored with the `store` method. + Useful to validate the model with EMA parameters without affecting the + original optimization process. Store the parameters before the + `copy_to` method. After validation (or model saving), use this to + restore the former parameters. + Args: + parameters: Iterable of `torch.nn.Parameter`; the parameters to be + updated with the stored parameters. + """ + for c_param, param in zip(self.collected_params, parameters): + param.data.copy_(c_param.data) diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__init__.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/__init__.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/__init__.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..f0f2db9adbe03f872cf78cfe7890e20a3d3b45f3 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/__init__.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/__init__.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/__init__.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..98fefd770808fa400d8e74b088ed3a03cba259b5 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/__init__.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/__init__.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/__init__.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..a00e9eb2741fdd4ce6fbe50e1351b3bb88b21b73 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/__init__.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/modules.cpython-310.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/modules.cpython-310.pyc new file mode 100644 index 0000000000000000000000000000000000000000..c03de9673812f97b1be31252cd88ddd4040ae263 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/modules.cpython-310.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/modules.cpython-311.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/modules.cpython-311.pyc new file mode 100644 index 0000000000000000000000000000000000000000..5524a54a81d3833a29cc4d4e47e149f74dae2da2 Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/modules.cpython-311.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/modules.cpython-39.pyc b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/modules.cpython-39.pyc new file mode 100644 index 0000000000000000000000000000000000000000..91e9295f47a03c6286a0100b39751a41570920aa Binary files /dev/null and b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/__pycache__/modules.cpython-39.pyc differ diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/modules.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/modules.py new file mode 100644 index 0000000000000000000000000000000000000000..6b51d5ed9737f017ba6a16c742e36aafc86958b0 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/modules/encoders/modules.py @@ -0,0 +1,325 @@ +import math +import torch +import torch.nn as nn +from torch.utils.checkpoint import checkpoint + +from transformers import T5Tokenizer, T5EncoderModel, CLIPTokenizer, CLIPTextModel +from ldm.modules.x_transformer import Encoder, TransformerWrapper # TODO: can we directly rely on lucidrains code and simply add this as a reuirement? --> test + +import open_clip +from ldm.util import default, count_params +from einops import repeat + +class AbstractEncoder(nn.Module): + def __init__(self): + super().__init__() + + def encode(self, *args, **kwargs): + raise NotImplementedError + + +class IdentityEncoder(AbstractEncoder): + + def encode(self, x): + return x + + +class ClassEmbedder(nn.Module): + def __init__(self, embed_dim, n_classes=1000, key='class', ucg_rate=0.1): + super().__init__() + self.key = key + self.embedding = nn.Embedding(n_classes, embed_dim) + self.n_classes = n_classes + self.ucg_rate = ucg_rate + + def forward(self, batch, key=None, disable_dropout=False): + if key is None: + key = self.key + # this is for use in crossattn + c = batch[key][:, None] + if self.ucg_rate > 0. and not disable_dropout: + mask = 1. - torch.bernoulli(torch.ones_like(c) * self.ucg_rate) + c = mask * c + (1-mask) * torch.ones_like(c)*(self.n_classes-1) + c = c.long() + c = self.embedding(c) + return c + + def get_unconditional_conditioning(self, bs, device="cuda"): + uc_class = self.n_classes - 1 # 1000 classes --> 0 ... 999, one extra class for ucg (class 1000) + uc = torch.ones((bs,), device=device) * uc_class + uc = {self.key: uc} + return uc + + +def disabled_train(self, mode=True): + """Overwrite model.train with this function to make sure train/eval mode + does not change anymore.""" + return self + +class BERTTokenizer(AbstractEncoder): + """ Uses a pretrained BERT tokenizer by huggingface. Vocab size: 30522 (?)""" + def __init__(self, device="cuda", vq_interface=True, max_length=77): + super().__init__() + from transformers import BertTokenizerFast # TODO: add to reuquirements + self.tokenizer = BertTokenizerFast.from_pretrained("bert-base-uncased") + self.device = device + self.vq_interface = vq_interface + self.max_length = max_length + + def forward(self, text): + batch_encoding = self.tokenizer(text, truncation=True, max_length=self.max_length, return_length=True, + return_overflowing_tokens=False, padding="max_length", return_tensors="pt") + tokens = batch_encoding["input_ids"].to(self.device) + return tokens + + @torch.no_grad() + def encode(self, text): + tokens = self(text) + if not self.vq_interface: + return tokens + return None, None, [None, None, tokens] + + def decode(self, text): + return text + +class BERTEmbedder(AbstractEncoder): + """Uses the BERT tokenizr model and add some transformer encoder layers""" + def __init__(self, n_embed, n_layer, vocab_size=30522, max_seq_len=77, + device="cuda",use_tokenizer=True, embedding_dropout=0.0): + super().__init__() + self.use_tknz_fn = use_tokenizer + if self.use_tknz_fn: + self.tknz_fn = BERTTokenizer(vq_interface=False, max_length=max_seq_len) + self.device = device + self.transformer = TransformerWrapper(num_tokens=vocab_size, max_seq_len=max_seq_len, + attn_layers=Encoder(dim=n_embed, depth=n_layer), + emb_dropout=embedding_dropout) + + def forward(self, text): + if self.use_tknz_fn: + tokens = self.tknz_fn(text)#.to(self.device) + else: + tokens = text + z = self.transformer(tokens, return_embeddings=True) + return z + + def encode(self, text): + # output of length 77 + return self(text) + +class MedBERTEmbedder(AbstractEncoder): + """Uses the BERT tokenizr model and add some transformer encoder layers""" + def __init__(self, n_embed, n_layer, vocab_size=30522, max_seq_len=77, + device="cuda",use_tokenizer=True, embedding_dropout=0.0): + super().__init__() + self.use_tknz_fn = use_tokenizer + if self.use_tknz_fn: + self.tknz_fn = BERTTokenizer(vq_interface=False, max_length=max_seq_len) + self.device = device + self.transformer = TransformerWrapper(num_tokens=vocab_size, max_seq_len=max_seq_len, + attn_layers=Encoder(dim=n_embed, depth=n_layer), + emb_dropout=embedding_dropout) + + def forward(self, text): + if self.use_tknz_fn: + tokens = self.tknz_fn(text)#.to(self.device) + else: + tokens = text + z = self.transformer(tokens, return_embeddings=True) + return z + + def encode(self, text): + # output of length 77 + return self(text) + +class FrozenT5Embedder(AbstractEncoder): + """Uses the T5 transformer encoder for text""" + def __init__(self, version="google/t5-v1_1-large", device="cuda", max_length=77, freeze=True): # others are google/t5-v1_1-xl and google/t5-v1_1-xxl + super().__init__() + self.tokenizer = T5Tokenizer.from_pretrained(version) + self.transformer = T5EncoderModel.from_pretrained(version) + self.device = device + self.max_length = max_length # TODO: typical value? + if freeze: + self.freeze() + + def freeze(self): + self.transformer = self.transformer.eval() + #self.train = disabled_train + for param in self.parameters(): + param.requires_grad = False + + def forward(self, text): + batch_encoding = self.tokenizer(text, truncation=True, max_length=self.max_length, return_length=True, + return_overflowing_tokens=False, padding="max_length", return_tensors="pt") + tokens = batch_encoding["input_ids"].to(self.device) + outputs = self.transformer(input_ids=tokens) + + z = outputs.last_hidden_state + return z + + def encode(self, text): + return self(text) + + +class FrozenCLIPEmbedder(AbstractEncoder): + """Uses the CLIP transformer encoder for text (from huggingface)""" + LAYERS = [ + "last", + "pooled", + "hidden" + ] + def __init__(self, version="openai/clip-vit-large-patch14", device="cuda", max_length=77, + freeze=True, layer="last", layer_idx=None): # clip-vit-base-patch32 + super().__init__() + assert layer in self.LAYERS + self.tokenizer = CLIPTokenizer.from_pretrained(version) + self.transformer = CLIPTextModel.from_pretrained(version) + self.device = device + self.max_length = max_length + if freeze: + self.freeze() + self.layer = layer + self.layer_idx = layer_idx + if layer == "hidden": + assert layer_idx is not None + assert 0 <= abs(layer_idx) <= 12 + + def freeze(self): + self.transformer = self.transformer.eval() + #self.train = disabled_train + for param in self.parameters(): + param.requires_grad = False + + def forward(self, text): + batch_encoding = self.tokenizer(text, truncation=True, max_length=self.max_length, return_length=True, + return_overflowing_tokens=False, padding="max_length", return_tensors="pt") + tokens = batch_encoding["input_ids"].to(self.device) + outputs = self.transformer(input_ids=tokens, output_hidden_states=self.layer=="hidden") + if self.layer == "last": + z = outputs.last_hidden_state + elif self.layer == "pooled": + z = outputs.pooler_output[:, None, :] + else: + z = outputs.hidden_states[self.layer_idx] + return z + + def encode(self, text): + return self(text) + + +class FrozenOpenCLIPEmbedder(AbstractEncoder): + """ + Uses the OpenCLIP transformer encoder for text + """ + LAYERS = [ + #"pooled", + "last", + "penultimate" + ] + def __init__(self, arch="ViT-H-14", version="laion2b_s32b_b79k", device="cuda", max_length=77, + freeze=True, layer="last"): + super().__init__() + assert layer in self.LAYERS + model, _, _ = open_clip.create_model_and_transforms(arch, device=torch.device('cpu'), pretrained=version) + del model.visual + self.model = model + + self.device = device + self.max_length = max_length + if freeze: + self.freeze() + self.layer = layer + if self.layer == "last": + self.layer_idx = 0 + elif self.layer == "penultimate": + self.layer_idx = 1 + else: + raise NotImplementedError() + + def freeze(self): + self.model = self.model.eval() + for param in self.parameters(): + param.requires_grad = False + + def forward(self, text): + tokens = open_clip.tokenize(text) + z = self.encode_with_transformer(tokens.to(self.device)) + return z + + def encode_with_transformer(self, text): + x = self.model.token_embedding(text) # [batch_size, n_ctx, d_model] + x = x + self.model.positional_embedding + x = x.permute(1, 0, 2) # NLD -> LND + x = self.text_transformer_forward(x, attn_mask=self.model.attn_mask) + x = x.permute(1, 0, 2) # LND -> NLD + x = self.model.ln_final(x) + return x + + def text_transformer_forward(self, x: torch.Tensor, attn_mask = None): + for i, r in enumerate(self.model.transformer.resblocks): + if i == len(self.model.transformer.resblocks) - self.layer_idx: + break + if self.model.transformer.grad_checkpointing and not torch.jit.is_scripting(): + x = checkpoint(r, x, attn_mask) + else: + x = r(x, attn_mask=attn_mask) + return x + + def encode(self, text): + return self(text) + + +class FrozenCLIPT5Encoder(AbstractEncoder): + def __init__(self, clip_version="openai/clip-vit-large-patch14", t5_version="google/t5-v1_1-xl", device="cuda", + clip_max_length=77, t5_max_length=77): + super().__init__() + self.clip_encoder = FrozenCLIPEmbedder(clip_version, device, max_length=clip_max_length) + self.t5_encoder = FrozenT5Embedder(t5_version, device, max_length=t5_max_length) + print(f"{self.clip_encoder.__class__.__name__} has {count_params(self.clip_encoder)*1.e-6:.2f} M parameters, " + f"{self.t5_encoder.__class__.__name__} comes with {count_params(self.t5_encoder)*1.e-6:.2f} M params.") + + def encode(self, text): + return self(text) + + def forward(self, text): + clip_z = self.clip_encoder.encode(text) + t5_z = self.t5_encoder.encode(text) + return [clip_z, t5_z] + + +def timestep_embedding(timesteps, dim, max_period=10000, repeat_only=False): + """ + Create sinusoidal timestep embeddings. + :param timesteps: a 1-D Tensor of N indices, one per batch element. + These may be fractional. + :param dim: the dimension of the output. + :param max_period: controls the minimum frequency of the embeddings. + :return: an [N x dim] Tensor of positional embeddings. + """ + if not repeat_only: + half = dim // 2 + freqs = torch.exp( + -math.log(max_period) * torch.arange(start=0, end=half, dtype=torch.float32) / half + ).to(device=timesteps.device) + args = timesteps[:, None].float() * freqs[None] + embedding = torch.cat([torch.cos(args), torch.sin(args)], dim=-1) + if dim % 2: + embedding = torch.cat([embedding, torch.zeros_like(embedding[:, :1])], dim=-1) + else: + embedding = repeat(timesteps, 'b -> b d', d=dim) + return embedding + + +class PositionalEmbedder(AbstractEncoder): + def __init__(self, dim, max_period=10000, repeat_only=False): + super().__init__() + self.dim = dim + self.max_period = max_period + self.repeat_only = repeat_only + + def encode(self, positional_id): + return self(positional_id) + + def forward(self, positional_id): + return timestep_embedding(1000*positional_id, self.dim, self.max_period, self.repeat_only) \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/modules/x_transformer.py b/Abnormal-CT-Generation-MultiDisease/ldm/modules/x_transformer.py new file mode 100644 index 0000000000000000000000000000000000000000..5fc15bf9cfe0111a910e7de33d04ffdec3877576 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/modules/x_transformer.py @@ -0,0 +1,641 @@ +"""shout-out to https://github.com/lucidrains/x-transformers/tree/main/x_transformers""" +import torch +from torch import nn, einsum +import torch.nn.functional as F +from functools import partial +from inspect import isfunction +from collections import namedtuple +from einops import rearrange, repeat, reduce + +# constants + +DEFAULT_DIM_HEAD = 64 + +Intermediates = namedtuple('Intermediates', [ + 'pre_softmax_attn', + 'post_softmax_attn' +]) + +LayerIntermediates = namedtuple('Intermediates', [ + 'hiddens', + 'attn_intermediates' +]) + + +class AbsolutePositionalEmbedding(nn.Module): + def __init__(self, dim, max_seq_len): + super().__init__() + self.emb = nn.Embedding(max_seq_len, dim) + self.init_() + + def init_(self): + nn.init.normal_(self.emb.weight, std=0.02) + + def forward(self, x): + n = torch.arange(x.shape[1], device=x.device) + return self.emb(n)[None, :, :] + + +class FixedPositionalEmbedding(nn.Module): + def __init__(self, dim): + super().__init__() + inv_freq = 1. / (10000 ** (torch.arange(0, dim, 2).float() / dim)) + self.register_buffer('inv_freq', inv_freq) + + def forward(self, x, seq_dim=1, offset=0): + t = torch.arange(x.shape[seq_dim], device=x.device).type_as(self.inv_freq) + offset + sinusoid_inp = torch.einsum('i , j -> i j', t, self.inv_freq) + emb = torch.cat((sinusoid_inp.sin(), sinusoid_inp.cos()), dim=-1) + return emb[None, :, :] + + +# helpers + +def exists(val): + return val is not None + + +def default(val, d): + if exists(val): + return val + return d() if isfunction(d) else d + + +def always(val): + def inner(*args, **kwargs): + return val + return inner + + +def not_equals(val): + def inner(x): + return x != val + return inner + + +def equals(val): + def inner(x): + return x == val + return inner + + +def max_neg_value(tensor): + return -torch.finfo(tensor.dtype).max + + +# keyword argument helpers + +def pick_and_pop(keys, d): + values = list(map(lambda key: d.pop(key), keys)) + return dict(zip(keys, values)) + + +def group_dict_by_key(cond, d): + return_val = [dict(), dict()] + for key in d.keys(): + match = bool(cond(key)) + ind = int(not match) + return_val[ind][key] = d[key] + return (*return_val,) + + +def string_begins_with(prefix, str): + return str.startswith(prefix) + + +def group_by_key_prefix(prefix, d): + return group_dict_by_key(partial(string_begins_with, prefix), d) + + +def groupby_prefix_and_trim(prefix, d): + kwargs_with_prefix, kwargs = group_dict_by_key(partial(string_begins_with, prefix), d) + kwargs_without_prefix = dict(map(lambda x: (x[0][len(prefix):], x[1]), tuple(kwargs_with_prefix.items()))) + return kwargs_without_prefix, kwargs + + +# classes +class Scale(nn.Module): + def __init__(self, value, fn): + super().__init__() + self.value = value + self.fn = fn + + def forward(self, x, **kwargs): + x, *rest = self.fn(x, **kwargs) + return (x * self.value, *rest) + + +class Rezero(nn.Module): + def __init__(self, fn): + super().__init__() + self.fn = fn + self.g = nn.Parameter(torch.zeros(1)) + + def forward(self, x, **kwargs): + x, *rest = self.fn(x, **kwargs) + return (x * self.g, *rest) + + +class ScaleNorm(nn.Module): + def __init__(self, dim, eps=1e-5): + super().__init__() + self.scale = dim ** -0.5 + self.eps = eps + self.g = nn.Parameter(torch.ones(1)) + + def forward(self, x): + norm = torch.norm(x, dim=-1, keepdim=True) * self.scale + return x / norm.clamp(min=self.eps) * self.g + + +class RMSNorm(nn.Module): + def __init__(self, dim, eps=1e-8): + super().__init__() + self.scale = dim ** -0.5 + self.eps = eps + self.g = nn.Parameter(torch.ones(dim)) + + def forward(self, x): + norm = torch.norm(x, dim=-1, keepdim=True) * self.scale + return x / norm.clamp(min=self.eps) * self.g + + +class Residual(nn.Module): + def forward(self, x, residual): + return x + residual + + +class GRUGating(nn.Module): + def __init__(self, dim): + super().__init__() + self.gru = nn.GRUCell(dim, dim) + + def forward(self, x, residual): + gated_output = self.gru( + rearrange(x, 'b n d -> (b n) d'), + rearrange(residual, 'b n d -> (b n) d') + ) + + return gated_output.reshape_as(x) + + +# feedforward + +class GEGLU(nn.Module): + def __init__(self, dim_in, dim_out): + super().__init__() + self.proj = nn.Linear(dim_in, dim_out * 2) + + def forward(self, x): + x, gate = self.proj(x).chunk(2, dim=-1) + return x * F.gelu(gate) + + +class FeedForward(nn.Module): + def __init__(self, dim, dim_out=None, mult=4, glu=False, dropout=0.): + super().__init__() + inner_dim = int(dim * mult) + dim_out = default(dim_out, dim) + project_in = nn.Sequential( + nn.Linear(dim, inner_dim), + nn.GELU() + ) if not glu else GEGLU(dim, inner_dim) + + self.net = nn.Sequential( + project_in, + nn.Dropout(dropout), + nn.Linear(inner_dim, dim_out) + ) + + def forward(self, x): + return self.net(x) + + +# attention. +class Attention(nn.Module): + def __init__( + self, + dim, + dim_head=DEFAULT_DIM_HEAD, + heads=8, + causal=False, + mask=None, + talking_heads=False, + sparse_topk=None, + use_entmax15=False, + num_mem_kv=0, + dropout=0., + on_attn=False + ): + super().__init__() + if use_entmax15: + raise NotImplementedError("Check out entmax activation instead of softmax activation!") + self.scale = dim_head ** -0.5 + self.heads = heads + self.causal = causal + self.mask = mask + + inner_dim = dim_head * heads + + self.to_q = nn.Linear(dim, inner_dim, bias=False) + self.to_k = nn.Linear(dim, inner_dim, bias=False) + self.to_v = nn.Linear(dim, inner_dim, bias=False) + self.dropout = nn.Dropout(dropout) + + # talking heads + self.talking_heads = talking_heads + if talking_heads: + self.pre_softmax_proj = nn.Parameter(torch.randn(heads, heads)) + self.post_softmax_proj = nn.Parameter(torch.randn(heads, heads)) + + # explicit topk sparse attention + self.sparse_topk = sparse_topk + + # entmax + #self.attn_fn = entmax15 if use_entmax15 else F.softmax + self.attn_fn = F.softmax + + # add memory key / values + self.num_mem_kv = num_mem_kv + if num_mem_kv > 0: + self.mem_k = nn.Parameter(torch.randn(heads, num_mem_kv, dim_head)) + self.mem_v = nn.Parameter(torch.randn(heads, num_mem_kv, dim_head)) + + # attention on attention + self.attn_on_attn = on_attn + self.to_out = nn.Sequential(nn.Linear(inner_dim, dim * 2), nn.GLU()) if on_attn else nn.Linear(inner_dim, dim) + + def forward( + self, + x, + context=None, + mask=None, + context_mask=None, + rel_pos=None, + sinusoidal_emb=None, + prev_attn=None, + mem=None + ): + b, n, _, h, talking_heads, device = *x.shape, self.heads, self.talking_heads, x.device + kv_input = default(context, x) + + q_input = x + k_input = kv_input + v_input = kv_input + + if exists(mem): + k_input = torch.cat((mem, k_input), dim=-2) + v_input = torch.cat((mem, v_input), dim=-2) + + if exists(sinusoidal_emb): + # in shortformer, the query would start at a position offset depending on the past cached memory + offset = k_input.shape[-2] - q_input.shape[-2] + q_input = q_input + sinusoidal_emb(q_input, offset=offset) + k_input = k_input + sinusoidal_emb(k_input) + + q = self.to_q(q_input) + k = self.to_k(k_input) + v = self.to_v(v_input) + + q, k, v = map(lambda t: rearrange(t, 'b n (h d) -> b h n d', h=h), (q, k, v)) + + input_mask = None + if any(map(exists, (mask, context_mask))): + q_mask = default(mask, lambda: torch.ones((b, n), device=device).bool()) + k_mask = q_mask if not exists(context) else context_mask + k_mask = default(k_mask, lambda: torch.ones((b, k.shape[-2]), device=device).bool()) + q_mask = rearrange(q_mask, 'b i -> b () i ()') + k_mask = rearrange(k_mask, 'b j -> b () () j') + input_mask = q_mask * k_mask + + if self.num_mem_kv > 0: + mem_k, mem_v = map(lambda t: repeat(t, 'h n d -> b h n d', b=b), (self.mem_k, self.mem_v)) + k = torch.cat((mem_k, k), dim=-2) + v = torch.cat((mem_v, v), dim=-2) + if exists(input_mask): + input_mask = F.pad(input_mask, (self.num_mem_kv, 0), value=True) + + dots = einsum('b h i d, b h j d -> b h i j', q, k) * self.scale + mask_value = max_neg_value(dots) + + if exists(prev_attn): + dots = dots + prev_attn + + pre_softmax_attn = dots + + if talking_heads: + dots = einsum('b h i j, h k -> b k i j', dots, self.pre_softmax_proj).contiguous() + + if exists(rel_pos): + dots = rel_pos(dots) + + if exists(input_mask): + dots.masked_fill_(~input_mask, mask_value) + del input_mask + + if self.causal: + i, j = dots.shape[-2:] + r = torch.arange(i, device=device) + mask = rearrange(r, 'i -> () () i ()') < rearrange(r, 'j -> () () () j') + mask = F.pad(mask, (j - i, 0), value=False) + dots.masked_fill_(mask, mask_value) + del mask + + if exists(self.sparse_topk) and self.sparse_topk < dots.shape[-1]: + top, _ = dots.topk(self.sparse_topk, dim=-1) + vk = top[..., -1].unsqueeze(-1).expand_as(dots) + mask = dots < vk + dots.masked_fill_(mask, mask_value) + del mask + + attn = self.attn_fn(dots, dim=-1) + post_softmax_attn = attn + + attn = self.dropout(attn) + + if talking_heads: + attn = einsum('b h i j, h k -> b k i j', attn, self.post_softmax_proj).contiguous() + + out = einsum('b h i j, b h j d -> b h i d', attn, v) + out = rearrange(out, 'b h n d -> b n (h d)') + + intermediates = Intermediates( + pre_softmax_attn=pre_softmax_attn, + post_softmax_attn=post_softmax_attn + ) + + return self.to_out(out), intermediates + + +class AttentionLayers(nn.Module): + def __init__( + self, + dim, + depth, + heads=8, + causal=False, + cross_attend=False, + only_cross=False, + use_scalenorm=False, + use_rmsnorm=False, + use_rezero=False, + rel_pos_num_buckets=32, + rel_pos_max_distance=128, + position_infused_attn=False, + custom_layers=None, + sandwich_coef=None, + par_ratio=None, + residual_attn=False, + cross_residual_attn=False, + macaron=False, + pre_norm=True, + gate_residual=False, + **kwargs + ): + super().__init__() + ff_kwargs, kwargs = groupby_prefix_and_trim('ff_', kwargs) + attn_kwargs, _ = groupby_prefix_and_trim('attn_', kwargs) + + dim_head = attn_kwargs.get('dim_head', DEFAULT_DIM_HEAD) + + self.dim = dim + self.depth = depth + self.layers = nn.ModuleList([]) + + self.has_pos_emb = position_infused_attn + self.pia_pos_emb = FixedPositionalEmbedding(dim) if position_infused_attn else None + self.rotary_pos_emb = always(None) + + assert rel_pos_num_buckets <= rel_pos_max_distance, 'number of relative position buckets must be less than the relative position max distance' + self.rel_pos = None + + self.pre_norm = pre_norm + + self.residual_attn = residual_attn + self.cross_residual_attn = cross_residual_attn + + norm_class = ScaleNorm if use_scalenorm else nn.LayerNorm + norm_class = RMSNorm if use_rmsnorm else norm_class + norm_fn = partial(norm_class, dim) + + norm_fn = nn.Identity if use_rezero else norm_fn + branch_fn = Rezero if use_rezero else None + + if cross_attend and not only_cross: + default_block = ('a', 'c', 'f') + elif cross_attend and only_cross: + default_block = ('c', 'f') + else: + default_block = ('a', 'f') + + if macaron: + default_block = ('f',) + default_block + + if exists(custom_layers): + layer_types = custom_layers + elif exists(par_ratio): + par_depth = depth * len(default_block) + assert 1 < par_ratio <= par_depth, 'par ratio out of range' + default_block = tuple(filter(not_equals('f'), default_block)) + par_attn = par_depth // par_ratio + depth_cut = par_depth * 2 // 3 # 2 / 3 attention layer cutoff suggested by PAR paper + par_width = (depth_cut + depth_cut // par_attn) // par_attn + assert len(default_block) <= par_width, 'default block is too large for par_ratio' + par_block = default_block + ('f',) * (par_width - len(default_block)) + par_head = par_block * par_attn + layer_types = par_head + ('f',) * (par_depth - len(par_head)) + elif exists(sandwich_coef): + assert sandwich_coef > 0 and sandwich_coef <= depth, 'sandwich coefficient should be less than the depth' + layer_types = ('a',) * sandwich_coef + default_block * (depth - sandwich_coef) + ('f',) * sandwich_coef + else: + layer_types = default_block * depth + + self.layer_types = layer_types + self.num_attn_layers = len(list(filter(equals('a'), layer_types))) + + for layer_type in self.layer_types: + if layer_type == 'a': + layer = Attention(dim, heads=heads, causal=causal, **attn_kwargs) + elif layer_type == 'c': + layer = Attention(dim, heads=heads, **attn_kwargs) + elif layer_type == 'f': + layer = FeedForward(dim, **ff_kwargs) + layer = layer if not macaron else Scale(0.5, layer) + else: + raise Exception(f'invalid layer type {layer_type}') + + if isinstance(layer, Attention) and exists(branch_fn): + layer = branch_fn(layer) + + if gate_residual: + residual_fn = GRUGating(dim) + else: + residual_fn = Residual() + + self.layers.append(nn.ModuleList([ + norm_fn(), + layer, + residual_fn + ])) + + def forward( + self, + x, + context=None, + mask=None, + context_mask=None, + mems=None, + return_hiddens=False + ): + hiddens = [] + intermediates = [] + prev_attn = None + prev_cross_attn = None + + mems = mems.copy() if exists(mems) else [None] * self.num_attn_layers + + for ind, (layer_type, (norm, block, residual_fn)) in enumerate(zip(self.layer_types, self.layers)): + is_last = ind == (len(self.layers) - 1) + + if layer_type == 'a': + hiddens.append(x) + layer_mem = mems.pop(0) + + residual = x + + if self.pre_norm: + x = norm(x) + + if layer_type == 'a': + out, inter = block(x, mask=mask, sinusoidal_emb=self.pia_pos_emb, rel_pos=self.rel_pos, + prev_attn=prev_attn, mem=layer_mem) + elif layer_type == 'c': + out, inter = block(x, context=context, mask=mask, context_mask=context_mask, prev_attn=prev_cross_attn) + elif layer_type == 'f': + out = block(x) + + x = residual_fn(out, residual) + + if layer_type in ('a', 'c'): + intermediates.append(inter) + + if layer_type == 'a' and self.residual_attn: + prev_attn = inter.pre_softmax_attn + elif layer_type == 'c' and self.cross_residual_attn: + prev_cross_attn = inter.pre_softmax_attn + + if not self.pre_norm and not is_last: + x = norm(x) + + if return_hiddens: + intermediates = LayerIntermediates( + hiddens=hiddens, + attn_intermediates=intermediates + ) + + return x, intermediates + + return x + + +class Encoder(AttentionLayers): + def __init__(self, **kwargs): + assert 'causal' not in kwargs, 'cannot set causality on encoder' + super().__init__(causal=False, **kwargs) + + + +class TransformerWrapper(nn.Module): + def __init__( + self, + *, + num_tokens, + max_seq_len, + attn_layers, + emb_dim=None, + max_mem_len=0., + emb_dropout=0., + num_memory_tokens=None, + tie_embedding=False, + use_pos_emb=True + ): + super().__init__() + assert isinstance(attn_layers, AttentionLayers), 'attention layers must be one of Encoder or Decoder' + + dim = attn_layers.dim + emb_dim = default(emb_dim, dim) + + self.max_seq_len = max_seq_len + self.max_mem_len = max_mem_len + self.num_tokens = num_tokens + + self.token_emb = nn.Embedding(num_tokens, emb_dim) + self.pos_emb = AbsolutePositionalEmbedding(emb_dim, max_seq_len) if ( + use_pos_emb and not attn_layers.has_pos_emb) else always(0) + self.emb_dropout = nn.Dropout(emb_dropout) + + self.project_emb = nn.Linear(emb_dim, dim) if emb_dim != dim else nn.Identity() + self.attn_layers = attn_layers + self.norm = nn.LayerNorm(dim) + + self.init_() + + self.to_logits = nn.Linear(dim, num_tokens) if not tie_embedding else lambda t: t @ self.token_emb.weight.t() + + # memory tokens (like [cls]) from Memory Transformers paper + num_memory_tokens = default(num_memory_tokens, 0) + self.num_memory_tokens = num_memory_tokens + if num_memory_tokens > 0: + self.memory_tokens = nn.Parameter(torch.randn(num_memory_tokens, dim)) + + # let funnel encoder know number of memory tokens, if specified + if hasattr(attn_layers, 'num_memory_tokens'): + attn_layers.num_memory_tokens = num_memory_tokens + + def init_(self): + nn.init.normal_(self.token_emb.weight, std=0.02) + + def forward( + self, + x, + return_embeddings=False, + mask=None, + return_mems=False, + return_attn=False, + mems=None, + **kwargs + ): + b, n, device, num_mem = *x.shape, x.device, self.num_memory_tokens + x = self.token_emb(x) + x += self.pos_emb(x) + x = self.emb_dropout(x) + + x = self.project_emb(x) + + if num_mem > 0: + mem = repeat(self.memory_tokens, 'n d -> b n d', b=b) + x = torch.cat((mem, x), dim=1) + + # auto-handle masking after appending memory tokens + if exists(mask): + mask = F.pad(mask, (num_mem, 0), value=True) + + x, intermediates = self.attn_layers(x, mask=mask, mems=mems, return_hiddens=True, **kwargs) + x = self.norm(x) + + mem, x = x[:, :num_mem], x[:, num_mem:] + + out = self.to_logits(x) if not return_embeddings else x + + if return_mems: + hiddens = intermediates.hiddens + new_mems = list(map(lambda pair: torch.cat(pair, dim=-2), zip(mems, hiddens))) if exists(mems) else hiddens + new_mems = list(map(lambda t: t[..., -self.max_mem_len:, :].detach(), new_mems)) + return out, new_mems + + if return_attn: + attn_maps = list(map(lambda t: t.post_softmax_attn, intermediates.attn_intermediates)) + return out, attn_maps + + return out + diff --git a/Abnormal-CT-Generation-MultiDisease/ldm/util.py b/Abnormal-CT-Generation-MultiDisease/ldm/util.py new file mode 100644 index 0000000000000000000000000000000000000000..45cb050ece6f401a22dde098ce3f1ff663c5eb6a --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/ldm/util.py @@ -0,0 +1,197 @@ +import importlib + +import torch +from torch import optim +import numpy as np + +from inspect import isfunction +from PIL import Image, ImageDraw, ImageFont + + +def log_txt_as_img(wh, xc, size=10): + # wh a tuple of (width, height) + # xc a list of captions to plot + b = len(xc) + txts = list() + for bi in range(b): + txt = Image.new("RGB", wh, color="white") + draw = ImageDraw.Draw(txt) + font = ImageFont.truetype('font/DejaVuSans.ttf', size=size) + nc = int(40 * (wh[0] / 256)) + lines = "\n".join(xc[bi][start:start + nc] for start in range(0, len(xc[bi]), nc)) + + try: + draw.text((0, 0), lines, fill="black", font=font) + except UnicodeEncodeError: + print("Cant encode string for logging. Skipping.") + + txt = np.array(txt).transpose(2, 0, 1) / 127.5 - 1.0 + txts.append(txt) + txts = np.stack(txts) + txts = torch.tensor(txts) + return txts + + +def ismap(x): + if not isinstance(x, torch.Tensor): + return False + return (len(x.shape) == 4) and (x.shape[1] > 3) + + +def isimage(x): + if not isinstance(x,torch.Tensor): + return False + return (len(x.shape) == 4) and (x.shape[1] == 3 or x.shape[1] == 1) + + +def exists(x): + return x is not None + + +def default(val, d): + if exists(val): + return val + return d() if isfunction(d) else d + + +def mean_flat(tensor): + """ + https://github.com/openai/guided-diffusion/blob/27c20a8fab9cb472df5d6bdd6c8d11c8f430b924/guided_diffusion/nn.py#L86 + Take the mean over all non-batch dimensions. + """ + return tensor.mean(dim=list(range(1, len(tensor.shape)))) + + +def count_params(model, verbose=False): + total_params = sum(p.numel() for p in model.parameters()) + if verbose: + print(f"{model.__class__.__name__} has {total_params*1.e-6:.2f} M params.") + return total_params + + +def instantiate_from_config(config): + if not "target" in config: + if config == '__is_first_stage__': + return None + elif config == "__is_unconditional__": + return None + raise KeyError("Expected key `target` to instantiate.") + return get_obj_from_str(config["target"])(**config.get("params", dict())) + + +def get_obj_from_str(string, reload=False): + module, cls = string.rsplit(".", 1) + if reload: + module_imp = importlib.import_module(module) + importlib.reload(module_imp) + return getattr(importlib.import_module(module, package=None), cls) + + +class AdamWwithEMAandWings(optim.Optimizer): + # credit to https://gist.github.com/crowsonkb/65f7265353f403714fce3b2595e0b298 + def __init__(self, params, lr=1.e-3, betas=(0.9, 0.999), eps=1.e-8, # TODO: check hyperparameters before using + weight_decay=1.e-2, amsgrad=False, ema_decay=0.9999, # ema decay to match previous code + ema_power=1., param_names=()): + """AdamW that saves EMA versions of the parameters.""" + if not 0.0 <= lr: + raise ValueError("Invalid learning rate: {}".format(lr)) + if not 0.0 <= eps: + raise ValueError("Invalid epsilon value: {}".format(eps)) + if not 0.0 <= betas[0] < 1.0: + raise ValueError("Invalid beta parameter at index 0: {}".format(betas[0])) + if not 0.0 <= betas[1] < 1.0: + raise ValueError("Invalid beta parameter at index 1: {}".format(betas[1])) + if not 0.0 <= weight_decay: + raise ValueError("Invalid weight_decay value: {}".format(weight_decay)) + if not 0.0 <= ema_decay <= 1.0: + raise ValueError("Invalid ema_decay value: {}".format(ema_decay)) + defaults = dict(lr=lr, betas=betas, eps=eps, + weight_decay=weight_decay, amsgrad=amsgrad, ema_decay=ema_decay, + ema_power=ema_power, param_names=param_names) + super().__init__(params, defaults) + + def __setstate__(self, state): + super().__setstate__(state) + for group in self.param_groups: + group.setdefault('amsgrad', False) + + @torch.no_grad() + def step(self, closure=None): + """Performs a single optimization step. + Args: + closure (callable, optional): A closure that reevaluates the model + and returns the loss. + """ + loss = None + if closure is not None: + with torch.enable_grad(): + loss = closure() + + for group in self.param_groups: + params_with_grad = [] + grads = [] + exp_avgs = [] + exp_avg_sqs = [] + ema_params_with_grad = [] + state_sums = [] + max_exp_avg_sqs = [] + state_steps = [] + amsgrad = group['amsgrad'] + beta1, beta2 = group['betas'] + ema_decay = group['ema_decay'] + ema_power = group['ema_power'] + + for p in group['params']: + if p.grad is None: + continue + params_with_grad.append(p) + if p.grad.is_sparse: + raise RuntimeError('AdamW does not support sparse gradients') + grads.append(p.grad) + + state = self.state[p] + + # State initialization + if len(state) == 0: + state['step'] = 0 + # Exponential moving average of gradient values + state['exp_avg'] = torch.zeros_like(p, memory_format=torch.preserve_format) + # Exponential moving average of squared gradient values + state['exp_avg_sq'] = torch.zeros_like(p, memory_format=torch.preserve_format) + if amsgrad: + # Maintains max of all exp. moving avg. of sq. grad. values + state['max_exp_avg_sq'] = torch.zeros_like(p, memory_format=torch.preserve_format) + # Exponential moving average of parameter values + state['param_exp_avg'] = p.detach().float().clone() + + exp_avgs.append(state['exp_avg']) + exp_avg_sqs.append(state['exp_avg_sq']) + ema_params_with_grad.append(state['param_exp_avg']) + + if amsgrad: + max_exp_avg_sqs.append(state['max_exp_avg_sq']) + + # update the steps for each param group update + state['step'] += 1 + # record the step after step update + state_steps.append(state['step']) + + optim._functional.adamw(params_with_grad, + grads, + exp_avgs, + exp_avg_sqs, + max_exp_avg_sqs, + state_steps, + amsgrad=amsgrad, + beta1=beta1, + beta2=beta2, + lr=group['lr'], + weight_decay=group['weight_decay'], + eps=group['eps'], + maximize=False) + + cur_ema_decay = min(ema_decay, 1 - state['step'] ** -ema_power) + for param, ema_param in zip(params_with_grad, ema_params_with_grad): + ema_param.mul_(cur_ema_decay).add_(param.float(), alpha=1 - cur_ema_decay) + + return loss \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000045.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000045.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..372c11f44afaaa300e0d8c06c99de0fcbfa1ebb3 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000045.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:922ab870073373a42c943aecae0af63a3f5538c0a13752d267aa8f12d2c91eba +size 7325150815 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000047.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000047.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..782937f4c0f0956f55144d7619f54b24a0378238 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000047.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f73b5765efb8329ed0977fb8c633b64e74f041982015a2e3326861ac369b1c9 +size 7325150815 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000057.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000057.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..52a04f041279ccd43f421faea61967b50fe86cd0 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000057.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5cb75c60e5a4c40d72f3802628edcd4074b048eafa3d3e3e6af0468d35658ed +size 7325151006 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000064.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000064.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..2b8aa7f52d617d678e37e3cfddc948cb34d368ab --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000064.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cb9e8f7a407e7e6d63c05a441878a2eb127cf5e5cb96b54701f5bc48a60924ef +size 7325158367 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000065.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000065.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..5b92425851022c3dae5676a4a4053bc7448ec1f0 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000065.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26731036ad9a305fe69373cf623e0ec73fbfe0560372209633e332d08127c18f +size 7325158367 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000068.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000068.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..2add6a613877fc8e29c2dfb9e907394bb141d030 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000068.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ebfc711b4176192699e7c309fa051de962eb3e627a318baa4056c74027086fe8 +size 7325158558 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000175.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000175.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..aa16b5b2505784858c1b1280a151bd9af5632efc --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000175.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:748e199f5efba68fc36c130814708c2be39155233d4f187c604154e92551097c +size 7325158367 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000294.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000294.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..b57ebc7415c20a34b0736b7cd93414a3150579b2 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000294.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ec8182ec8505b7cd642a54f00047dc5fbf4f79b2123eb517ad380f43ded3ada +size 7325158367 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000317.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000317.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..51a51e9b2e43a44569b69630d16347eafa14fb9c --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000317.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4f9eb3a56c6c42665afc7bb4778a5f873566efe564ccc8a5394bc2169bc28f9d +size 7325158622 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/last.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/last.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..0d5b288de1fcb851a48ce0f66de9112463743e35 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/checkpoints/last.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:553d31fed01c121cf3d3c3bb52bc9f003e22360953ae13f99ecc116e9a0ae524 +size 7325158622 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-29T11-50-24-lightning.yaml b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-29T11-50-24-lightning.yaml new file mode 100644 index 0000000000000000000000000000000000000000..af576c4f4222ea130206951d1e6274cd32183a05 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-29T11-50-24-lightning.yaml @@ -0,0 +1,12 @@ +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: false + trainer: + benchmark: true + accelerator: ddp + gpus: 0,1,2,3, diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-29T11-50-24-project.yaml b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-29T11-50-24-project.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0818df4c2ab0bb444ad6babe59b9bce57001c083 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-29T11-50-24-project.yaml @@ -0,0 +1,58 @@ +model: + base_learning_rate: 5.0e-07 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.012 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + conditioning_key: crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: true + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 32 + out_channels: 16 + model_channels: 224 + attention_resolutions: + - 8 + - 4 + - 2 + num_res_blocks: 2 + channel_mult: + - 1 + - 2 + - 3 + - 4 + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: true + legacy: false +data: + target: main.DataModuleFromConfig + params: + batch_size: 8 + num_workers: 12 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: /sd/shuhan/CT-RATE/dataset/train_fixed + csv_file: /sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: /sd/shuhan/CT-RATE/dataset/valid_fixed + csv_file: /sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv + labels: /sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-31T23-48-10-lightning.yaml b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-31T23-48-10-lightning.yaml new file mode 100644 index 0000000000000000000000000000000000000000..1b0338aead6c797df75ee541d54de45b0d63accf --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-31T23-48-10-lightning.yaml @@ -0,0 +1,13 @@ +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: false + trainer: + benchmark: true + accelerator: ddp + gpus: 0,1,2,3, + resume_from_checkpoint: ./logs/full_ct_2d_with_body_mask2/checkpoints/last.ckpt diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-31T23-48-10-project.yaml b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-31T23-48-10-project.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0818df4c2ab0bb444ad6babe59b9bce57001c083 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-10-31T23-48-10-project.yaml @@ -0,0 +1,58 @@ +model: + base_learning_rate: 5.0e-07 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.012 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + conditioning_key: crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: true + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 32 + out_channels: 16 + model_channels: 224 + attention_resolutions: + - 8 + - 4 + - 2 + num_res_blocks: 2 + channel_mult: + - 1 + - 2 + - 3 + - 4 + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: true + legacy: false +data: + target: main.DataModuleFromConfig + params: + batch_size: 8 + num_workers: 12 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: /sd/shuhan/CT-RATE/dataset/train_fixed + csv_file: /sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: /sd/shuhan/CT-RATE/dataset/valid_fixed + csv_file: /sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv + labels: /sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T20-43-27-lightning.yaml b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T20-43-27-lightning.yaml new file mode 100644 index 0000000000000000000000000000000000000000..1b0338aead6c797df75ee541d54de45b0d63accf --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T20-43-27-lightning.yaml @@ -0,0 +1,13 @@ +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: false + trainer: + benchmark: true + accelerator: ddp + gpus: 0,1,2,3, + resume_from_checkpoint: ./logs/full_ct_2d_with_body_mask2/checkpoints/last.ckpt diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T20-43-27-project.yaml b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T20-43-27-project.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0818df4c2ab0bb444ad6babe59b9bce57001c083 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T20-43-27-project.yaml @@ -0,0 +1,58 @@ +model: + base_learning_rate: 5.0e-07 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.012 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + conditioning_key: crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: true + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 32 + out_channels: 16 + model_channels: 224 + attention_resolutions: + - 8 + - 4 + - 2 + num_res_blocks: 2 + channel_mult: + - 1 + - 2 + - 3 + - 4 + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: true + legacy: false +data: + target: main.DataModuleFromConfig + params: + batch_size: 8 + num_workers: 12 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: /sd/shuhan/CT-RATE/dataset/train_fixed + csv_file: /sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: /sd/shuhan/CT-RATE/dataset/valid_fixed + csv_file: /sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv + labels: /sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T21-49-33-lightning.yaml b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T21-49-33-lightning.yaml new file mode 100644 index 0000000000000000000000000000000000000000..1b0338aead6c797df75ee541d54de45b0d63accf --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T21-49-33-lightning.yaml @@ -0,0 +1,13 @@ +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: false + trainer: + benchmark: true + accelerator: ddp + gpus: 0,1,2,3, + resume_from_checkpoint: ./logs/full_ct_2d_with_body_mask2/checkpoints/last.ckpt diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T21-49-33-project.yaml b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T21-49-33-project.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0818df4c2ab0bb444ad6babe59b9bce57001c083 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/configs/2025-12-11T21-49-33-project.yaml @@ -0,0 +1,58 @@ +model: + base_learning_rate: 5.0e-07 + target: ldm.models.diffusion.ddpm.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.012 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + conditioning_key: crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: true + unet_config: + target: ldm.modules.diffusionmodules.openaimodel.UNetModel + params: + image_size: 64 + in_channels: 32 + out_channels: 16 + model_channels: 224 + attention_resolutions: + - 8 + - 4 + - 2 + num_res_blocks: 2 + channel_mult: + - 1 + - 2 + - 3 + - 4 + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: true + legacy: false +data: + target: main.DataModuleFromConfig + params: + batch_size: 8 + num_workers: 12 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: /sd/shuhan/CT-RATE/dataset/train_fixed + csv_file: /sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: /sd/shuhan/CT-RATE/dataset/valid_fixed + csv_file: /sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv + labels: /sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/meta.experiment b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/meta.experiment new file mode 100644 index 0000000000000000000000000000000000000000..966fd6ab95c3f0feb98bf55c3177d86b7f6a0ed4 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/meta.experiment @@ -0,0 +1 @@ +{"name": "testtube", "version": 0, "tags_path": "logs/full_ct_2d_with_body_mask2/testtube/version_0/meta_tags.csv", "metrics_path": "logs/full_ct_2d_with_body_mask2/testtube/version_0/metrics.csv", "autosave": false, "description": null, "created_at": "2025-10-29 11:51:02.139781", "exp_hash": "testtube_v0"} \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/meta_tags.csv b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/meta_tags.csv new file mode 100644 index 0000000000000000000000000000000000000000..942a47139f86778b1f3e5ad923104dd038b1ce88 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/meta_tags.csv @@ -0,0 +1 @@ +key,value diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/metrics.csv b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/metrics.csv new file mode 100644 index 0000000000000000000000000000000000000000..ff28246ed34dbc2c96bcd7e8d35a7cc9a164fe6b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/metrics.csv @@ -0,0 +1,266 @@ +train/loss_simple_step,train/loss_vlb_step,train/loss_step,global_step,epoch,created_at,val/loss_simple,val/loss_vlb,val/loss,val/loss_simple_ema,val/loss_vlb_ema,val/loss_ema,train/loss_simple_epoch,train/loss_vlb_epoch,train/loss_epoch +0.9296233654022217,0.006323201581835747,0.9296233654022217,49.0,0,2025-10-29 12:11:03.056162,,,,,,,,, +0.8281739354133606,0.004871342796832323,0.8281739354133606,99.0,0,2025-10-29 12:28:30.156085,,,,,,,,, +,,,,0,2025-10-29 12:36:58.714496,0.7679132223129272,0.006631164811551571,0.7679132223129272,0.795319139957428,0.005986308213323355,0.795319139957428,,, +,,,,0,2025-10-29 12:38:08.533766,,,,,,,0.8970698118209839,0.006654718890786171,0.8970698118209839 +0.7189406156539917,0.0050783297047019005,0.7189406156539917,149.0,1,2025-10-29 12:52:46.365409,,,,,,,,, +0.6408004760742188,0.011028442531824112,0.6408004760742188,199.0,1,2025-10-29 13:06:36.965762,,,,,,,,, +,,,,1,2025-10-29 13:31:24.374366,0.5617077946662903,0.005384430754929781,0.5617077946662903,0.5932433009147644,0.006243112497031689,0.5932433009147644,,, +,,,,1,2025-10-29 13:32:36.893176,,,,,,,0.6623785495758057,0.005684825591742992,0.6623785495758057 +0.586790144443512,0.0043334392830729485,0.586790144443512,249.0,2,2025-10-29 13:36:28.736433,,,,,,,,, +0.5013716220855713,0.008361107669770718,0.5013716220855713,299.0,2,2025-10-29 13:54:48.533396,,,,,,,,, +0.4522360563278198,0.002237793290987611,0.4522360563278198,349.0,2,2025-10-29 14:17:17.282556,,,,,,,,, +,,,,2,2025-10-29 14:27:38.488567,0.4392277002334595,0.0036598395090550184,0.4392277002334595,0.46507906913757324,0.004609440453350544,0.46507906913757324,,, +,,,,2,2025-10-29 14:28:50.944745,,,,,,,0.5023209452629089,0.004780743736773729,0.5023209452629089 +0.47378242015838623,0.002638439415022731,0.47378242015838623,399.0,3,2025-10-29 14:43:45.774334,,,,,,,,, +0.4107520282268524,0.004982852842658758,0.4107520282268524,449.0,3,2025-10-29 15:11:14.284757,,,,,,,,, +,,,,3,2025-10-29 15:28:31.454060,0.3602713942527771,0.004258666653186083,0.3602713942527771,0.3877120018005371,0.004027978517115116,0.3877120018005371,,, +,,,,3,2025-10-29 15:29:48.786950,,,,,,,0.40727266669273376,0.004576192703098059,0.40727266669273376 +0.3878153860569,0.0036686162929981947,0.3878153860569,499.0,4,2025-10-29 15:36:41.561881,,,,,,,,, +0.36420130729675293,0.004634419921785593,0.36420130729675293,549.0,4,2025-10-29 15:55:19.281325,,,,,,,,, +0.36977508664131165,0.003418943379074335,0.36977508664131165,599.0,4,2025-10-29 16:12:26.473709,,,,,,,,, +,,,,4,2025-10-29 16:24:31.026774,0.31022876501083374,0.0042220731265842915,0.31022876501083374,0.3345106840133667,0.004281933885067701,0.3345106840133667,,, +,,,,4,2025-10-29 16:25:42.748867,,,,,,,0.3540625274181366,0.004087630193680525,0.3540625274181366 +0.33438998460769653,0.00454533938318491,0.33438998460769653,649.0,5,2025-10-29 16:44:10.428543,,,,,,,,, +0.33210253715515137,0.0028151606675237417,0.33210253715515137,699.0,5,2025-10-29 17:01:00.901962,,,,,,,,, +,,,,5,2025-10-29 17:20:00.149763,0.2798500657081604,0.0032463266979902983,0.2798500657081604,0.29333487153053284,0.004873638041317463,0.29333487153053284,,, +,,,,5,2025-10-29 17:21:15.389853,,,,,,,0.3148147463798523,0.004111896269023418,0.3148147463798523 +0.35072797536849976,0.00388258695602417,0.35072797536849976,749.0,6,2025-10-29 17:27:33.719750,,,,,,,,, +0.24924306571483612,0.0014764994848519564,0.24924306571483612,799.0,6,2025-10-29 17:47:48.885386,,,,,,,,, +0.2528529465198517,0.0015789450844749808,0.2528529465198517,849.0,6,2025-10-29 18:04:50.192515,,,,,,,,, +,,,,6,2025-10-29 18:14:27.874327,0.26337435841560364,0.003603486344218254,0.26337435841560364,0.26693058013916016,0.0035697564017027617,0.26693058013916016,,, +,,,,6,2025-10-29 18:15:46.811349,,,,,,,0.2944440245628357,0.0037530148401856422,0.2944440245628357 +0.27340778708457947,0.00582224503159523,0.27340778708457947,899.0,7,2025-10-29 18:33:53.584383,,,,,,,,, +0.28887906670570374,0.003398980712518096,0.28887906670570374,949.0,7,2025-10-29 18:53:33.878821,,,,,,,,, +,,,,7,2025-10-29 19:19:41.801627,0.2672083079814911,0.004076921846717596,0.2672083079814911,0.2703196108341217,0.003315872745588422,0.2703196108341217,,, +,,,,7,2025-10-29 19:21:03.700098,,,,,,,0.2839674651622772,0.003763349261134863,0.2839674651622772 +0.272748738527298,0.003843760583549738,0.272748738527298,999.0,8,2025-10-29 19:29:06.009543,,,,,,,,, +0.2691429853439331,0.005615465808659792,0.2691429853439331,1049.0,8,2025-10-29 19:49:29.526271,,,,,,,,, +0.24354851245880127,0.007105142343789339,0.24354851245880127,1099.0,8,2025-10-29 20:06:49.969435,,,,,,,,, +,,,,8,2025-10-29 20:21:39.052003,0.2425854355096817,0.003921613097190857,0.2425854355096817,0.25546276569366455,0.0026105670258402824,0.25546276569366455,,, +,,,,8,2025-10-29 20:23:04.437335,,,,,,,0.26924511790275574,0.0037121372297406197,0.26924511790275574 +0.3125245273113251,0.007336942479014397,0.3125245273113251,1149.0,9,2025-10-29 20:41:09.024224,,,,,,,,, +0.2563961446285248,0.0014403603272512555,0.2563961446285248,1199.0,9,2025-10-29 21:01:59.059433,,,,,,,,, +,,,,9,2025-10-29 21:17:16.946877,0.23004987835884094,0.0032737962901592255,0.23004987835884094,0.24542896449565887,0.0034312971401959658,0.24542896449565887,,, +,,,,9,2025-10-29 21:18:58.777073,,,,,,,0.26397550106048584,0.0033445933368057013,0.26397550106048584 +0.24616673588752747,0.001515016658231616,0.24616673588752747,1249.0,10,2025-10-29 21:29:31.119407,,,,,,,,, +0.24127079546451569,0.0016851146938279271,0.24127079546451569,1299.0,10,2025-10-29 21:57:52.226231,,,,,,,,, +0.2658845782279968,0.0023453673347830772,0.2658845782279968,1349.0,10,2025-10-29 22:18:18.203685,,,,,,,,, +,,,,10,2025-10-29 22:27:26.437601,0.23198620975017548,0.0019819126464426517,0.23198620975017548,0.24453595280647278,0.0034107728861272335,0.24453595280647278,,, +,,,,10,2025-10-29 22:28:47.340490,,,,,,,0.2582567036151886,0.0035111908800899982,0.2582567036151886 +0.28123244643211365,0.002631989074870944,0.28123244643211365,1399.0,11,2025-10-29 22:47:24.503284,,,,,,,,, +0.2466992735862732,0.003127565374597907,0.2466992735862732,1449.0,11,2025-10-29 23:06:06.803588,,,,,,,,, +,,,,11,2025-10-29 23:23:03.210780,0.2342105358839035,0.0024233844596892595,0.2342105358839035,0.24127016961574554,0.003316126298159361,0.24127016961574554,,, +,,,,11,2025-10-29 23:24:17.581270,,,,,,,0.2579015791416168,0.0032941331155598164,0.2579015791416168 +0.23656822741031647,0.0018284755060449243,0.23656822741031647,1499.0,12,2025-10-29 23:33:28.729382,,,,,,,,, +0.2646838128566742,0.0022894495632499456,0.2646838128566742,1549.0,12,2025-10-29 23:54:24.374393,,,,,,,,, +0.25223544239997864,0.005738350562751293,0.25223544239997864,1599.0,12,2025-10-30 00:12:08.426485,,,,,,,,, +,,,,12,2025-10-30 00:23:01.043822,0.2286405861377716,0.004141286481171846,0.2286405861377716,0.2371658980846405,0.0036332528106868267,0.2371658980846405,,, +,,,,12,2025-10-30 00:30:13.585162,,,,,,,0.25340914726257324,0.003686076495796442,0.25340914726257324 +0.23667141795158386,0.002570581156760454,0.23667141795158386,1649.0,13,2025-10-30 00:53:26.263072,,,,,,,,, +0.2717689871788025,0.0026461207307875156,0.2717689871788025,1699.0,13,2025-10-30 01:11:25.806949,,,,,,,,, +,,,,13,2025-10-30 01:32:43.102819,0.23149444162845612,0.002944433130323887,0.23149444162845612,0.2341085970401764,0.003791593248024583,0.2341085970401764,,, +,,,,13,2025-10-30 01:34:28.386989,,,,,,,0.24941818416118622,0.003576261457055807,0.24941818416118622 +0.24516935646533966,0.0023914645425975323,0.24516935646533966,1749.0,14,2025-10-30 01:43:30.642911,,,,,,,,, +0.19261077046394348,0.002049604430794716,0.19261077046394348,1799.0,14,2025-10-30 02:05:34.731161,,,,,,,,, +0.23765718936920166,0.0019328462658450007,0.23765718936920166,1849.0,14,2025-10-30 02:28:34.143935,,,,,,,,, +,,,,14,2025-10-30 02:36:29.255907,0.23493097722530365,0.0034332049544900656,0.23493097722530365,0.23130792379379272,0.0027580177411437035,0.23130792379379272,,, +,,,,14,2025-10-30 02:37:46.116211,,,,,,,0.2494358867406845,0.003559673437848687,0.2494358867406845 +0.27682697772979736,0.0069122957065701485,0.27682697772979736,1899.0,15,2025-10-30 02:56:39.727544,,,,,,,,, +0.20615443587303162,0.006138400174677372,0.20615443587303162,1949.0,15,2025-10-30 03:15:25.819776,,,,,,,,, +,,,,15,2025-10-30 03:30:46.316386,0.22277352213859558,0.002379580633714795,0.22277352213859558,0.23270413279533386,0.0026512891054153442,0.23270413279533386,,, +,,,,15,2025-10-30 03:32:04.171880,,,,,,,0.24486637115478516,0.003621782874688506,0.24486637115478516 +0.2512393295764923,0.004151768982410431,0.2512393295764923,1999.0,16,2025-10-30 03:41:25.252317,,,,,,,,, +0.23365385830402374,0.00524112256243825,0.23365385830402374,2049.0,16,2025-10-30 04:00:28.217370,,,,,,,,, +0.2478683441877365,0.004409522283822298,0.2478683441877365,2099.0,16,2025-10-30 04:20:40.446880,,,,,,,,, +,,,,16,2025-10-30 04:29:06.280702,0.2398727685213089,0.004424940329045057,0.2398727685213089,0.22807131707668304,0.002846908289939165,0.22807131707668304,,, +,,,,16,2025-10-30 04:30:23.097050,,,,,,,0.2410329431295395,0.003360839793458581,0.2410329431295395 +0.25572669506073,0.0029490625020116568,0.25572669506073,2149.0,17,2025-10-30 04:49:44.675419,,,,,,,,, +0.23860302567481995,0.0014977427199482918,0.23860302567481995,2199.0,17,2025-10-30 05:14:31.029158,,,,,,,,, +,,,,17,2025-10-30 05:28:02.840465,0.21904465556144714,0.0028708649333566427,0.21904465556144714,0.22741959989070892,0.00311101577244699,0.22741959989070892,,, +,,,,17,2025-10-30 05:29:21.388228,,,,,,,0.24862274527549744,0.0032155343797057867,0.24862274527549744 +0.198785200715065,0.0020594720263034105,0.198785200715065,2249.0,18,2025-10-30 05:43:18.837883,,,,,,,,, +0.24799464643001556,0.0014812115114182234,0.24799464643001556,2299.0,18,2025-10-30 06:04:51.807528,,,,,,,,, +0.22709259390830994,0.0024709582794457674,0.22709259390830994,2349.0,18,2025-10-30 06:20:50.377173,,,,,,,,, +,,,,18,2025-10-30 06:27:24.211345,0.21540644764900208,0.002696717856451869,0.21540644764900208,0.2300853133201599,0.003538967575877905,0.2300853133201599,,, +,,,,18,2025-10-30 06:28:39.654385,,,,,,,0.24370311200618744,0.0034940377809107304,0.24370311200618744 +0.30100423097610474,0.009679161943495274,0.30100423097610474,2399.0,19,2025-10-30 06:51:27.118157,,,,,,,,, +0.26833170652389526,0.003783529857173562,0.26833170652389526,2449.0,19,2025-10-30 07:16:24.860314,,,,,,,,, +,,,,19,2025-10-30 07:36:44.330437,0.22714892029762268,0.003336181165650487,0.22714892029762268,0.21924936771392822,0.0037840150762349367,0.21924936771392822,,, +,,,,19,2025-10-30 07:38:01.978561,,,,,,,0.24842216074466705,0.0038228563498705626,0.24842216074466705 +0.22938354313373566,0.00561569444835186,0.22938354313373566,2499.0,20,2025-10-30 07:49:05.461577,,,,,,,,, +0.21908794343471527,0.005676914006471634,0.21908794343471527,2549.0,20,2025-10-30 08:10:38.461082,,,,,,,,, +0.25299856066703796,0.005995957180857658,0.25299856066703796,2599.0,20,2025-10-30 08:29:48.326676,,,,,,,,, +,,,,20,2025-10-30 08:35:19.974023,0.22261664271354675,0.0034261182881891727,0.22261664271354675,0.22938796877861023,0.002186909783631563,0.22938796877861023,,, +,,,,20,2025-10-30 08:35:57.522361,,,,,,,0.24082615971565247,0.003545391606166959,0.24082615971565247 +0.20897428691387177,0.003495168872177601,0.20897428691387177,2649.0,21,2025-10-30 08:59:49.833192,,,,,,,,, +0.16003933548927307,0.0015886797336861491,0.16003933548927307,2699.0,21,2025-10-30 09:18:04.451866,,,,,,,,, +,,,,21,2025-10-30 09:33:58.619583,0.2297607809305191,0.004535730462521315,0.2297607809305191,0.2219724953174591,0.002383551560342312,0.2219724953174591,,, +,,,,21,2025-10-30 09:35:18.017338,,,,,,,0.23668107390403748,0.0034774180967360735,0.23668107390403748 +0.2506035566329956,0.0028152186423540115,0.2506035566329956,2749.0,22,2025-10-30 09:49:50.909737,,,,,,,,, +0.2154092639684677,0.005283506121486425,0.2154092639684677,2799.0,22,2025-10-30 10:04:45.415471,,,,,,,,, +0.22911591827869415,0.0017922456609085202,0.22911591827869415,2849.0,22,2025-10-30 10:31:33.449080,,,,,,,,, +,,,,22,2025-10-30 10:37:06.614819,0.22115395963191986,0.003931779880076647,0.22115395963191986,0.21938160061836243,0.003991886507719755,0.21938160061836243,,, +,,,,22,2025-10-30 10:38:20.872593,,,,,,,0.23611989617347717,0.003175091464072466,0.23611989617347717 +0.24227288365364075,0.003051182022318244,0.24227288365364075,2899.0,23,2025-10-30 11:02:08.867841,,,,,,,,, +0.2677517533302307,0.00162382866255939,0.2677517533302307,2949.0,23,2025-10-30 11:21:02.885531,,,,,,,,, +,,,,23,2025-10-30 11:36:43.724498,0.20675253868103027,0.00229751318693161,0.20675253868103027,0.2204841673374176,0.002853142097592354,0.2204841673374176,,, +,,,,23,2025-10-30 11:37:58.286557,,,,,,,0.24062952399253845,0.0034607064444571733,0.24062952399253845 +0.24417747557163239,0.0017898952355608344,0.24417747557163239,2999.0,24,2025-10-30 11:50:08.928729,,,,,,,,, +0.22950434684753418,0.0030233138240873814,0.22950434684753418,3049.0,24,2025-10-30 12:15:57.529183,,,,,,,,, +0.2182105928659439,0.001885387348011136,0.2182105928659439,3099.0,24,2025-10-30 12:33:43.134951,,,,,,,,, +,,,,24,2025-10-30 12:38:30.861715,0.21597324311733246,0.0032159064430743456,0.21597324311733246,0.21791286766529083,0.002107167150825262,0.21791286766529083,,, +,,,,24,2025-10-30 12:39:54.798904,,,,,,,0.23362766206264496,0.002823365619406104,0.23362766206264496 +0.26873117685317993,0.006025270093232393,0.26873117685317993,3149.0,25,2025-10-30 13:10:39.465467,,,,,,,,, +0.21960954368114471,0.007399900816380978,0.21960954368114471,3199.0,25,2025-10-30 13:31:28.065038,,,,,,,,, +,,,,25,2025-10-30 13:43:00.202049,0.22108255326747894,0.0036868618335574865,0.22108255326747894,0.22149977087974548,0.002542107366025448,0.22149977087974548,,, +,,,,25,2025-10-30 13:43:40.940573,,,,,,,0.23694966733455658,0.003174298908561468,0.23694966733455658 +0.22263962030410767,0.0026535990182310343,0.22263962030410767,3249.0,26,2025-10-30 13:58:26.625213,,,,,,,,, +0.26136910915374756,0.006215991452336311,0.26136910915374756,3299.0,26,2025-10-30 14:16:42.560901,,,,,,,,, +,,,,26,2025-10-30 14:43:18.992565,0.21837516129016876,0.001982538728043437,0.21837516129016876,0.21662861108779907,0.0037863668985664845,0.21662861108779907,,, +,,,,26,2025-10-30 14:44:36.428263,,,,,,,0.2325274795293808,0.003351994091644883,0.2325274795293808 +0.22863183915615082,0.005831521470099688,0.22863183915615082,3349.0,27,2025-10-30 14:53:25.411975,,,,,,,,, +0.2592438757419586,0.01066174078732729,0.2592438757419586,3399.0,27,2025-10-30 15:10:05.567321,,,,,,,,, +0.1860283762216568,0.0012482332531362772,0.1860283762216568,3449.0,27,2025-10-30 15:30:13.471089,,,,,,,,, +,,,,27,2025-10-30 15:42:34.937892,0.2267545610666275,0.004541011527180672,0.2267545610666275,0.20642119646072388,0.003683265531435609,0.20642119646072388,,, +,,,,27,2025-10-30 15:43:53.353052,,,,,,,0.22923508286476135,0.003146728267893195,0.22923508286476135 +0.227031409740448,0.0013427281519398093,0.227031409740448,3499.0,28,2025-10-30 16:03:06.429431,,,,,,,,, +0.22054354846477509,0.002438734984025359,0.22054354846477509,3549.0,28,2025-10-30 16:30:07.722293,,,,,,,,, +,,,,28,2025-10-30 16:49:38.951459,0.2055787742137909,0.00371151277795434,0.2055787742137909,0.21275989711284637,0.0031822044402360916,0.21275989711284637,,, +,,,,28,2025-10-30 16:50:55.180094,,,,,,,0.23200319707393646,0.0031994821038097143,0.23200319707393646 +0.21952249109745026,0.0016677769599482417,0.21952249109745026,3599.0,29,2025-10-30 16:55:23.946621,,,,,,,,, +0.2363344430923462,0.0023444409016519785,0.2363344430923462,3649.0,29,2025-10-30 17:15:46.641338,,,,,,,,, +0.24896633625030518,0.0016278392868116498,0.24896633625030518,3699.0,29,2025-10-30 17:36:18.277294,,,,,,,,, +,,,,29,2025-10-30 17:48:57.542834,0.21938572824001312,0.0032521625980734825,0.21938572824001312,0.21594606339931488,0.0036499665584415197,0.21594606339931488,,, +,,,,29,2025-10-30 17:50:19.623455,,,,,,,0.23112188279628754,0.0029570558108389378,0.23112188279628754 +0.22653837502002716,0.0017968390602618456,0.22653837502002716,3749.0,30,2025-10-30 18:11:31.112789,,,,,,,,, +0.22266003489494324,0.005777828395366669,0.22266003489494324,3799.0,30,2025-10-30 18:32:40.898482,,,,,,,,, +,,,,30,2025-10-30 18:52:42.770173,0.2141987532377243,0.003242508275434375,0.2141987532377243,0.2150110900402069,0.002087157918140292,0.2150110900402069,,, +,,,,30,2025-10-30 18:53:54.646498,,,,,,,0.22588486969470978,0.0031507157254964113,0.22588486969470978 +0.22812144458293915,0.00290929083712399,0.22812144458293915,3849.0,31,2025-10-30 19:00:50.335464,,,,,,,,, +0.25777173042297363,0.002727518556639552,0.25777173042297363,3899.0,31,2025-10-30 19:24:22.979874,,,,,,,,, +0.2229062169790268,0.006360833067446947,0.2229062169790268,3949.0,31,2025-10-30 19:42:48.299291,,,,,,,,, +,,,,31,2025-10-30 19:54:51.129930,0.21683548390865326,0.002589619252830744,0.21683548390865326,0.21726730465888977,0.0020801464561372995,0.21726730465888977,,, +,,,,31,2025-10-30 19:55:32.932454,,,,,,,0.2289019376039505,0.003483385546132922,0.2289019376039505 +0.2192588746547699,0.0024758120998740196,0.2192588746547699,3999.0,32,2025-10-30 20:15:31.906122,,,,,,,,, +0.178208589553833,0.0016813910333439708,0.178208589553833,4049.0,32,2025-10-30 20:36:35.014655,,,,,,,,, +,,,,32,2025-10-30 21:01:31.728791,0.21511530876159668,0.002838749438524246,0.21511530876159668,0.2082805335521698,0.003510303096845746,0.2082805335521698,,, +,,,,32,2025-10-30 21:02:52.022710,,,,,,,0.22992347180843353,0.0032973112538456917,0.22992347180843353 +0.21768833696842194,0.00499361427500844,0.21768833696842194,4099.0,33,2025-10-30 21:10:50.934632,,,,,,,,, +0.22544032335281372,0.005444412585347891,0.22544032335281372,4149.0,33,2025-10-30 21:29:56.794921,,,,,,,,, +0.2698274254798889,0.0055450331419706345,0.2698274254798889,4199.0,33,2025-10-30 21:49:45.360794,,,,,,,,, +,,,,33,2025-10-30 21:58:13.910493,0.20441296696662903,0.0021100130397826433,0.20441296696662903,0.21529658138751984,0.003804202890023589,0.21529658138751984,,, +,,,,33,2025-10-30 21:58:50.307895,,,,,,,0.23104754090309143,0.003552602604031563,0.23104754090309143 +0.22780591249465942,0.002641965402290225,0.22780591249465942,4249.0,34,2025-10-30 22:19:12.508904,,,,,,,,, +0.2027958184480667,0.0017950861947610974,0.2027958184480667,4299.0,34,2025-10-30 22:43:32.739464,,,,,,,,, +,,,,34,2025-10-30 22:59:59.045131,0.2066170573234558,0.0028461425099521875,0.2066170573234558,0.2115931510925293,0.002423579338937998,0.2115931510925293,,, +,,,,34,2025-10-30 23:01:14.803634,,,,,,,0.22710509598255157,0.0031352206133306026,0.22710509598255157 +0.20971471071243286,0.004889531526714563,0.20971471071243286,4349.0,35,2025-10-30 23:09:35.076712,,,,,,,,, +0.21438480913639069,0.0013743554009124637,0.21438480913639069,4399.0,35,2025-10-30 23:24:29.243213,,,,,,,,, +0.21629324555397034,0.003291000844910741,0.21629324555397034,4449.0,35,2025-10-30 23:44:25.761200,,,,,,,,, +,,,,35,2025-10-30 23:56:52.574253,0.21975187957286835,0.0032476726919412613,0.21975187957286835,0.21103797852993011,0.003471486968919635,0.21103797852993011,,, +,,,,35,2025-10-30 23:58:08.676979,,,,,,,0.22613899409770966,0.003175453981384635,0.22613899409770966 +0.17304928600788116,0.0016320133581757545,0.17304928600788116,4499.0,36,2025-10-31 00:16:18.628053,,,,,,,,, +0.25205299258232117,0.01252166461199522,0.25205299258232117,4549.0,36,2025-10-31 00:40:05.961474,,,,,,,,, +,,,,36,2025-10-31 01:00:27.603491,0.20506951212882996,0.0025937682949006557,0.20506951212882996,0.2072264403104782,0.003330542240291834,0.2072264403104782,,, +,,,,36,2025-10-31 01:01:46.553857,,,,,,,0.22714512050151825,0.003375378903001547,0.22714512050151825 +0.2290341705083847,0.005988272838294506,0.2290341705083847,4599.0,37,2025-10-31 01:10:06.557891,,,,,,,,, +0.18789014220237732,0.004212846979498863,0.18789014220237732,4649.0,37,2025-10-31 01:32:36.401524,,,,,,,,, +0.24077174067497253,0.005134338513016701,0.24077174067497253,4699.0,37,2025-10-31 01:54:51.145253,,,,,,,,, +,,,,37,2025-10-31 02:02:33.872958,0.20805473625659943,0.002602646593004465,0.20805473625659943,0.20798978209495544,0.003907087724655867,0.20798978209495544,,, +,,,,37,2025-10-31 02:03:55.199435,,,,,,,0.22662824392318726,0.003525898791849613,0.22662824392318726 +0.22744080424308777,0.00528167886659503,0.22744080424308777,4749.0,38,2025-10-31 02:24:24.579692,,,,,,,,, +0.196287140250206,0.005025492515414953,0.196287140250206,4799.0,38,2025-10-31 02:46:40.077767,,,,,,,,, +,,,,38,2025-10-31 02:59:33.054434,0.22454628348350525,0.0031233232002705336,0.22454628348350525,0.2121679186820984,0.003267036285251379,0.2121679186820984,,, +,,,,38,2025-10-31 03:00:13.062127,,,,,,,0.22436057031154633,0.003263920545578003,0.22436057031154633 +0.20297470688819885,0.0013933677691966295,0.20297470688819885,4849.0,39,2025-10-31 03:12:05.544412,,,,,,,,, +0.20565254986286163,0.0024946490302681923,0.20565254986286163,4899.0,39,2025-10-31 03:36:08.802213,,,,,,,,, +0.21251071989536285,0.0016136285848915577,0.21251071989536285,4949.0,39,2025-10-31 03:55:27.748897,,,,,,,,, +,,,,39,2025-10-31 04:01:35.296292,0.20398376882076263,0.003491020295768976,0.20398376882076263,0.2113586813211441,0.0030506751500070095,0.2113586813211441,,, +,,,,39,2025-10-31 04:05:17.090347,,,,,,,0.22474873065948486,0.0033030349295586348,0.22474873065948486 +0.22770699858665466,0.001932766055688262,0.22770699858665466,4999.0,40,2025-10-31 04:26:08.476400,,,,,,,,, +0.22545374929904938,0.002892774995416403,0.22545374929904938,5049.0,40,2025-10-31 04:44:03.588092,,,,,,,,, +,,,,40,2025-10-31 05:05:20.679201,0.19539599120616913,0.002694594208151102,0.19539599120616913,0.21683575212955475,0.0031977887265384197,0.21683575212955475,,, +,,,,40,2025-10-31 05:06:01.588863,,,,,,,0.22637802362442017,0.003069397760555148,0.22637802362442017 +0.19911836087703705,0.0016642383998259902,0.19911836087703705,5099.0,41,2025-10-31 05:20:41.257383,,,,,,,,, +0.22438298165798187,0.004367198329418898,0.22438298165798187,5149.0,41,2025-10-31 05:36:16.866668,,,,,,,,, +0.24129584431648254,0.002429724670946598,0.24129584431648254,5199.0,41,2025-10-31 06:03:35.113751,,,,,,,,, +,,,,41,2025-10-31 06:09:34.192774,0.19639553129673004,0.003032445441931486,0.19639553129673004,0.22136297821998596,0.0030363735277205706,0.22136297821998596,,, +,,,,41,2025-10-31 06:10:11.150995,,,,,,,0.2232801765203476,0.003343405667692423,0.2232801765203476 +0.177547425031662,0.0042420183308422565,0.177547425031662,5249.0,42,2025-10-31 06:32:33.777617,,,,,,,,, +0.23656702041625977,0.0020603893790394068,0.23656702041625977,5299.0,42,2025-10-31 06:49:46.566663,,,,,,,,, +,,,,42,2025-10-31 07:07:07.917440,0.1998533010482788,0.0025087648537009954,0.1998533010482788,0.21661460399627686,0.003867646213620901,0.21661460399627686,,, +,,,,42,2025-10-31 07:07:59.048603,,,,,,,0.22539298236370087,0.0033092282246798277,0.22539298236370087 +0.25021645426750183,0.0027356960345059633,0.25021645426750183,5349.0,43,2025-10-31 07:27:26.852576,,,,,,,,, +0.20807623863220215,0.005009129643440247,0.20807623863220215,5399.0,43,2025-10-31 07:54:08.433492,,,,,,,,, +0.19087569415569305,0.005038049537688494,0.19087569415569305,5449.0,43,2025-10-31 08:08:11.766709,,,,,,,,, +,,,,43,2025-10-31 08:13:46.883745,0.20934589207172394,0.0031197345815598965,0.20934589207172394,0.21359601616859436,0.0024802745319902897,0.21359601616859436,,, +,,,,43,2025-10-31 08:14:27.728581,,,,,,,0.22524568438529968,0.003488969523459673,0.22524568438529968 +0.2166738659143448,0.006048965267837048,0.2166738659143448,5499.0,44,2025-10-31 08:35:38.273463,,,,,,,,, +0.252515584230423,0.003182473825290799,0.252515584230423,5549.0,44,2025-10-31 08:51:22.477860,,,,,,,,, +,,,,44,2025-10-31 09:06:03.860699,0.19593138992786407,0.00230636284686625,0.19593138992786407,0.2115613967180252,0.0037585636600852013,0.2115613967180252,,, +,,,,44,2025-10-31 09:06:52.031434,,,,,,,0.2230238914489746,0.0033409292809665203,0.2230238914489746 +0.2030176818370819,0.001972342375665903,0.2030176818370819,5599.0,45,2025-10-31 09:18:55.882935,,,,,,,,, +0.18946802616119385,0.004204284865409136,0.18946802616119385,5649.0,45,2025-10-31 09:43:29.535515,,,,,,,,, +0.21312128007411957,0.0033382410183548927,0.21312128007411957,5699.0,45,2025-10-31 09:56:02.475216,,,,,,,,, +,,,,45,2025-10-31 10:02:43.219832,0.20043249428272247,0.0023931211326271296,0.20043249428272247,0.19249677658081055,0.001889495295472443,0.19249677658081055,,, +,,,,45,2025-10-31 10:03:53.834314,,,,,,,0.2219030261039734,0.003148444928228855,0.2219030261039734 +0.20678289234638214,0.0013146026758477092,0.20678289234638214,5749.0,46,2025-10-31 10:22:11.364129,,,,,,,,, +0.18922649323940277,0.0012968180235475302,0.18922649323940277,5799.0,46,2025-10-31 10:43:42.568487,,,,,,,,, +,,,,46,2025-10-31 10:58:50.788609,0.21386562287807465,0.0033775886986404657,0.21386562287807465,0.21102553606033325,0.0025549421552568674,0.21102553606033325,,, +,,,,46,2025-10-31 10:59:29.876734,,,,,,,0.22352808713912964,0.0030533280223608017,0.22352808713912964 +0.20661412179470062,0.002765120007097721,0.20661412179470062,5849.0,47,2025-10-31 11:13:46.180190,,,,,,,,, +0.2382931411266327,0.005485607776790857,0.2382931411266327,5899.0,47,2025-10-31 11:29:56.131851,,,,,,,,, +0.19238002598285675,0.0025387704372406006,0.19238002598285675,5949.0,47,2025-10-31 11:45:36.910604,,,,,,,,, +,,,,47,2025-10-31 11:50:41.999574,0.21567372977733612,0.003254869021475315,0.21567372977733612,0.18760235607624054,0.0019027725793421268,0.18760235607624054,,, +,,,,47,2025-10-31 11:51:58.231967,,,,,,,0.22252707183361053,0.0033082549925893545,0.22252707183361053 +0.21015755832195282,0.0020279556047171354,0.21015755832195282,5999.0,48,2025-10-31 12:17:56.189131,,,,,,,,, +0.20875734090805054,0.0018147804075852036,0.20875734090805054,6049.0,48,2025-10-31 12:37:58.568323,,,,,,,,, +,,,,48,2025-10-31 12:51:11.829794,0.20864909887313843,0.002253815531730652,0.20864909887313843,0.20605921745300293,0.0038162413984537125,0.20605921745300293,,, +,,,,48,2025-10-31 12:52:31.483456,,,,,,,0.22332511842250824,0.003063349984586239,0.22332511842250824 +0.24758237600326538,0.0013026355300098658,0.24758237600326538,6099.0,49,2025-10-31 13:06:13.446887,,,,,,,,, +0.21152086555957794,0.0020754693541675806,0.21152086555957794,6149.0,49,2025-10-31 13:28:07.024935,,,,,,,,, +0.20605452358722687,0.0018582966877147555,0.20605452358722687,6199.0,49,2025-10-31 13:48:15.749360,,,,,,,,, +,,,,49,2025-10-31 13:53:41.162100,0.2067992240190506,0.0027446758467704058,0.2067992240190506,0.19949448108673096,0.00255148159340024,0.19949448108673096,,, +,,,,49,2025-10-31 13:54:56.820132,,,,,,,0.22113505005836487,0.0032367072999477386,0.22113505005836487 +0.22032848000526428,0.0031147946137934923,0.22032848000526428,6249.0,50,2025-10-31 14:20:03.130879,,,,,,,,, +0.2355663925409317,0.005210744217038155,0.2355663925409317,6299.0,50,2025-10-31 14:37:53.448389,,,,,,,,, +,,,,50,2025-10-31 14:52:50.458857,0.208190456032753,0.0032868538983166218,0.208190456032753,0.21250352263450623,0.003964188043028116,0.21250352263450623,,, +,,,,50,2025-10-31 14:53:42.901755,,,,,,,0.2177804559469223,0.0032252499368041754,0.2177804559469223 +0.2303667813539505,0.003065535333007574,0.2303667813539505,6349.0,51,2025-10-31 15:09:26.325384,,,,,,,,, +0.26654693484306335,0.0026986305601894855,0.26654693484306335,6399.0,51,2025-10-31 15:24:52.189665,,,,,,,,, +,,,,51,2025-10-31 15:49:53.184062,0.2081526219844818,0.002816365100443363,0.2081526219844818,0.21803219616413116,0.003822055645287037,0.21803219616413116,,, +,,,,51,2025-10-31 15:50:41.298259,,,,,,,0.22049833834171295,0.0033831684850156307,0.22049833834171295 +0.1990116685628891,0.0019983595702797174,0.1990116685628891,6449.0,52,2025-10-31 15:56:30.570977,,,,,,,,, +0.21557800471782684,0.002123018493875861,0.21557800471782684,6499.0,52,2025-10-31 16:17:26.213601,,,,,,,,, +0.17005766928195953,0.001059115631505847,0.17005766928195953,6549.0,52,2025-10-31 16:39:01.250933,,,,,,,,, +,,,,52,2025-10-31 16:50:19.009503,0.1862957626581192,0.0022564749233424664,0.1862957626581192,0.20891061425209045,0.0021090093068778515,0.20891061425209045,,, +,,,,52,2025-10-31 16:50:58.546023,,,,,,,0.21719227731227875,0.003191030351445079,0.21719227731227875 +0.24668842554092407,0.008778073824942112,0.24668842554092407,6599.0,53,2025-10-31 17:02:53.430140,,,,,,,,, +0.21960757672786713,0.0018289779545739293,0.21960757672786713,6649.0,53,2025-10-31 17:28:08.739932,,,,,,,,, +,,,,53,2025-10-31 17:51:00.485995,0.20794512331485748,0.0034701870754361153,0.20794512331485748,0.19906273484230042,0.002674701390787959,0.19906273484230042,,, +,,,,53,2025-10-31 17:52:25.892639,,,,,,,0.2203526794910431,0.0031459343153983355,0.2203526794910431 +0.2343813180923462,0.0022501973435282707,0.2343813180923462,6699.0,54,2025-10-31 17:59:41.650111,,,,,,,,, +0.1740899533033371,0.0010131503222510219,0.1740899533033371,6749.0,54,2025-10-31 18:22:45.693850,,,,,,,,, +0.26018306612968445,0.0015751009341329336,0.26018306612968445,6799.0,54,2025-10-31 18:44:31.303640,,,,,,,,, +,,,,54,2025-10-31 18:56:21.122465,0.20394104719161987,0.0026149183977395296,0.20394104719161987,0.20133532583713531,0.0031456651631742716,0.20133532583713531,,, +,,,,54,2025-10-31 18:56:58.934596,,,,,,,0.22004981338977814,0.003538313554599881,0.22004981338977814 +0.23548120260238647,0.0028353105299174786,0.23548120260238647,6849.0,55,2025-10-31 19:16:10.411627,,,,,,,,, +0.2374141663312912,0.0023281110916286707,0.2374141663312912,6899.0,55,2025-10-31 19:36:34.019975,,,,,,,,, +,,,,55,2025-10-31 19:56:35.708646,0.20696601271629333,0.0035260573495179415,0.20696601271629333,0.20068830251693726,0.002969686407595873,0.20068830251693726,,, +,,,,55,2025-10-31 19:57:14.429289,,,,,,,0.21959204971790314,0.00306299957446754,0.21959204971790314 +0.25001609325408936,0.003766613081097603,0.25001609325408936,6949.0,56,2025-10-31 20:04:44.742371,,,,,,,,, +0.2154899686574936,0.0045380243100225925,0.2154899686574936,6999.0,56,2025-10-31 20:31:41.236030,,,,,,,,, +0.22485554218292236,0.005087059922516346,0.22485554218292236,7049.0,56,2025-10-31 20:52:01.367012,,,,,,,,, +,,,,56,2025-10-31 21:02:24.098850,0.2001667469739914,0.0033871023915708065,0.2001667469739914,0.20784378051757812,0.0034978347830474377,0.20784378051757812,,, +,,,,56,2025-10-31 21:03:10.254439,,,,,,,0.21949656307697296,0.003310347907245159,0.21949656307697296 +0.2322734147310257,0.00408165855333209,0.2322734147310257,7099.0,57,2025-10-31 21:19:14.030608,,,,,,,,, +0.268091082572937,0.0027338021900504827,0.268091082572937,7149.0,57,2025-10-31 21:35:05.360143,,,,,,,,, +,,,,57,2025-10-31 21:53:54.775803,0.19784800708293915,0.003482889151200652,0.19784800708293915,0.18930847942829132,0.0018115165876224637,0.18930847942829132,,, +,,,,57,2025-10-31 21:55:08.969922,,,,,,,0.21953430771827698,0.0031539297197014093,0.21953430771827698 +0.22436080873012543,0.002435861388221383,0.22436080873012543,7199.0,58,2025-10-31 22:05:03.753535,,,,,,,,, +0.23287639021873474,0.004109982866793871,0.23287639021873474,7249.0,58,2025-10-31 22:25:54.407017,,,,,,,,, +0.21701358258724213,0.0014892485924065113,0.21701358258724213,7299.0,58,2025-10-31 22:50:16.856560,,,,,,,,, +,,,,58,2025-10-31 22:58:42.848052,0.20171448588371277,0.003500481601804495,0.20171448588371277,0.1955210417509079,0.0031773578375577927,0.1955210417509079,,, +,,,,58,2025-10-31 22:59:36.863469,,,,,,,0.21968477964401245,0.003211363684386015,0.21968477964401245 +0.25497734546661377,0.002167260507121682,0.25497734546661377,7349.0,59,2025-10-31 23:24:41.352997,,,,,,,,, diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/tf/events.out.tfevents.1761738664.node-0.2018.0 b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/tf/events.out.tfevents.1761738664.node-0.2018.0 new file mode 100644 index 0000000000000000000000000000000000000000..0eba6b703d77f79f299d98dada1d4553d4d8195f --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_0/tf/events.out.tfevents.1761738664.node-0.2018.0 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:39622e4c17d664ea71ab18f6888458d3ccf03ddb49d3b53dd268656133c07e73 +size 73536 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/meta.experiment b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/meta.experiment new file mode 100644 index 0000000000000000000000000000000000000000..ab7f2dea8f0181265bd9db9aacf4e01c8ae9cc6f --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/meta.experiment @@ -0,0 +1 @@ +{"name": "testtube", "version": 1, "tags_path": "./logs/full_ct_2d_with_body_mask2/testtube/version_1/meta_tags.csv", "metrics_path": "./logs/full_ct_2d_with_body_mask2/testtube/version_1/metrics.csv", "autosave": false, "description": null, "created_at": "2025-10-31 23:49:12.098353", "exp_hash": "testtube_v1"} \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/meta_tags.csv b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/meta_tags.csv new file mode 100644 index 0000000000000000000000000000000000000000..942a47139f86778b1f3e5ad923104dd038b1ce88 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/meta_tags.csv @@ -0,0 +1 @@ +key,value diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/metrics.csv b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/metrics.csv new file mode 100644 index 0000000000000000000000000000000000000000..083a6c278950f00cde4b4152255d60dafab06da9 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/metrics.csv @@ -0,0 +1,69 @@ +train/loss_simple_step,train/loss_vlb_step,train/loss_step,global_step,epoch,created_at,val/loss_simple,val/loss_vlb,val/loss,val/loss_simple_ema,val/loss_vlb_ema,val/loss_ema,train/loss_simple_epoch,train/loss_vlb_epoch,train/loss_epoch +0.2528144121170044,0.0018980252789333463,0.2528144121170044,7349.0,59,2025-11-01 00:13:03.732593,,,,,,,,, +0.1767973005771637,0.010176529176533222,0.1767973005771637,7399.0,59,2025-11-01 00:31:47.621512,,,,,,,,, +,,,,59,2025-11-01 00:52:16.360207,0.20430797338485718,0.003277173498645425,0.20430797338485718,0.2037089616060257,0.0027820654213428497,0.2037089616060257,,, +,,,,59,2025-11-01 00:53:35.800012,,,,,,,0.21767187118530273,0.0032942185644060373,0.21767187118530273 +0.22581034898757935,0.00159752881154418,0.22581034898757935,7449.0,60,2025-11-01 01:02:24.755787,,,,,,,,, +0.2793352007865906,0.0057570310309529305,0.2793352007865906,7499.0,60,2025-11-01 01:22:27.426522,,,,,,,,, +0.2192530632019043,0.0016275221714749932,0.2192530632019043,7549.0,60,2025-11-01 01:44:56.601624,,,,,,,,, +,,,,60,2025-11-01 01:52:52.390586,0.20018501579761505,0.0031670057214796543,0.20018501579761505,0.197466641664505,0.003745360765606165,0.197466641664505,,, +,,,,60,2025-11-01 01:54:10.404244,,,,,,,0.22005334496498108,0.003323567332699895,0.22005334496498108 +0.2521190345287323,0.001974900718778372,0.2521190345287323,7599.0,61,2025-11-01 02:20:30.725150,,,,,,,,, +0.18525460362434387,0.004767692182213068,0.18525460362434387,7649.0,61,2025-11-01 02:41:20.877485,,,,,,,,, +,,,,61,2025-11-01 02:56:24.414063,0.20482131838798523,0.002364036161452532,0.20482131838798523,0.1979900300502777,0.003003865946084261,0.1979900300502777,,, +,,,,61,2025-11-01 02:57:52.010219,,,,,,,0.2215145230293274,0.0032075245399028063,0.2215145230293274 +0.21123705804347992,0.0069627054035663605,0.21123705804347992,7699.0,62,2025-11-01 03:08:40.009365,,,,,,,,, +0.2219223976135254,0.0030697009060531855,0.2219223976135254,7749.0,62,2025-11-01 03:27:57.379869,,,,,,,,, +0.20269761979579926,0.0023367071989923716,0.20269761979579926,7799.0,62,2025-11-01 03:45:15.569497,,,,,,,,, +,,,,62,2025-11-01 03:51:48.592231,0.20012739300727844,0.0030568765942007303,0.20012739300727844,0.20308159291744232,0.002979400334879756,0.20308159291744232,,, +,,,,62,2025-11-01 03:53:07.044813,,,,,,,0.21719981729984283,0.0033993571996688843,0.21719981729984283 +0.2724059522151947,0.009035617113113403,0.2724059522151947,7849.0,63,2025-11-01 04:09:10.957607,,,,,,,,, +0.20395712554454803,0.00228859088383615,0.20395712554454803,7899.0,63,2025-11-01 04:28:06.555983,,,,,,,,, +,,,,63,2025-11-01 04:47:45.310064,0.19651466608047485,0.0033523316960781813,0.19651466608047485,0.20060372352600098,0.0033844076097011566,0.20060372352600098,,, +,,,,63,2025-11-01 04:49:07.290352,,,,,,,0.21726645529270172,0.003235939657315612,0.21726645529270172 +0.18545232713222504,0.0015166677767410874,0.18545232713222504,7949.0,64,2025-11-01 04:55:59.803850,,,,,,,,, +0.20851458609104156,0.001372550381347537,0.20851458609104156,7999.0,64,2025-11-01 05:16:40.651786,,,,,,,,, +0.20708762109279633,0.0017805317183956504,0.20708762109279633,8049.0,64,2025-11-01 05:33:09.776173,,,,,,,,, +,,,,64,2025-11-01 05:39:40.532927,0.1929323971271515,0.002596598118543625,0.1929323971271515,0.19221965968608856,0.004123140126466751,0.19221965968608856,,, +,,,,64,2025-11-01 05:41:02.852757,,,,,,,0.2169915735721588,0.003402301575988531,0.2169915735721588 +0.17428293824195862,0.001305932179093361,0.17428293824195862,8099.0,65,2025-11-01 05:58:18.939986,,,,,,,,, +0.2653758227825165,0.003527512075379491,0.2653758227825165,8149.0,65,2025-11-01 06:18:11.648750,,,,,,,,, +,,,,65,2025-11-01 06:39:16.825505,0.19373531639575958,0.003090188605710864,0.19373531639575958,0.18818242847919464,0.0030541731975972652,0.18818242847919464,,, +,,,,65,2025-11-01 06:40:43.307914,,,,,,,0.22060172259807587,0.003264746395871043,0.22060172259807587 +0.21315062046051025,0.0013636243529617786,0.21315062046051025,8199.0,66,2025-11-01 06:51:14.992588,,,,,,,,, +0.21464188396930695,0.0017898862715810537,0.21464188396930695,8249.0,66,2025-11-01 07:10:12.244621,,,,,,,,, +0.2110358327627182,0.0014108879258856177,0.2110358327627182,8299.0,66,2025-11-01 07:26:25.646623,,,,,,,,, +,,,,66,2025-11-01 07:33:51.454264,0.20764249563217163,0.003275624243542552,0.20764249563217163,0.2042253464460373,0.002878699218854308,0.2042253464460373,,, +,,,,66,2025-11-01 07:34:36.308309,,,,,,,0.21532569825649261,0.003185619367286563,0.21532569825649261 +0.20791596174240112,0.0019471908453851938,0.20791596174240112,8349.0,67,2025-11-01 07:54:32.288182,,,,,,,,, +0.22680804133415222,0.006100279279053211,0.22680804133415222,8399.0,67,2025-11-01 08:09:23.872061,,,,,,,,, +,,,,67,2025-11-01 08:25:13.146516,0.19245970249176025,0.0034407719504088163,0.19245970249176025,0.19955945014953613,0.0022430960088968277,0.19955945014953613,,, +,,,,67,2025-11-01 08:25:55.926189,,,,,,,0.21592141687870026,0.0032640162389725447,0.21592141687870026 +0.25974592566490173,0.005675915628671646,0.25974592566490173,8449.0,68,2025-11-01 08:36:15.209537,,,,,,,,, +0.22462530434131622,0.001774882199242711,0.22462530434131622,8499.0,68,2025-11-01 08:57:01.110007,,,,,,,,, +0.2398023009300232,0.00252410676330328,0.2398023009300232,8549.0,68,2025-11-01 09:12:35.973068,,,,,,,,, +,,,,68,2025-11-01 09:18:30.642341,0.18586504459381104,0.0028868294321000576,0.18586504459381104,0.1961793303489685,0.0030164378695189953,0.1961793303489685,,, +,,,,68,2025-11-01 09:20:01.975910,,,,,,,0.211045041680336,0.0029456024058163166,0.211045041680336 +0.24370793998241425,0.003980314824730158,0.24370793998241425,8599.0,69,2025-11-01 09:40:25.076343,,,,,,,,, +0.2008693367242813,0.001406615599989891,0.2008693367242813,8649.0,69,2025-11-01 10:03:38.379714,,,,,,,,, +,,,,69,2025-11-01 10:17:39.785816,0.1898963451385498,0.0017196027329191566,0.1898963451385498,0.19950956106185913,0.0030365455895662308,0.19950956106185913,,, +,,,,69,2025-11-01 10:18:25.121899,,,,,,,0.21126027405261993,0.003152078716084361,0.21126027405261993 +0.22750666737556458,0.010260222479701042,0.22750666737556458,8699.0,70,2025-11-01 10:32:11.755893,,,,,,,,, +0.16151560842990875,0.0012950872769579291,0.16151560842990875,8749.0,70,2025-11-01 10:50:41.318605,,,,,,,,, +0.19212134182453156,0.004972812719643116,0.19212134182453156,8799.0,70,2025-11-01 11:10:27.681524,,,,,,,,, +,,,,70,2025-11-01 11:16:03.998461,0.19570305943489075,0.0021485439501702785,0.19570305943489075,0.20002183318138123,0.0029859065543860197,0.20002183318138123,,, +,,,,70,2025-11-01 11:16:46.365176,,,,,,,0.21646550297737122,0.0029693162068724632,0.21646550297737122 +0.20639009773731232,0.002088228240609169,0.20639009773731232,8849.0,71,2025-11-01 11:44:20.203729,,,,,,,,, +0.2145238071680069,0.0019112748559564352,0.2145238071680069,8899.0,71,2025-11-01 11:59:00.934178,,,,,,,,, +,,,,71,2025-11-01 12:11:28.801526,0.192085400223732,0.003766048699617386,0.192085400223732,0.19883187115192413,0.0032622788567095995,0.19883187115192413,,, +,,,,71,2025-11-01 12:12:12.494767,,,,,,,0.21590422093868256,0.0033424929715692997,0.21590422093868256 +0.19740724563598633,0.0011529044713824987,0.19740724563598633,8949.0,72,2025-11-01 12:25:01.149172,,,,,,,,, +0.24304567277431488,0.009725806303322315,0.24304567277431488,8999.0,72,2025-11-01 12:40:55.865429,,,,,,,,, +0.24507573246955872,0.0027379528619349003,0.24507573246955872,9049.0,72,2025-11-01 12:59:48.275669,,,,,,,,, +,,,,72,2025-11-01 13:06:17.484024,0.1972641795873642,0.0026982049457728863,0.1972641795873642,0.19744321703910828,0.0033088093623518944,0.19744321703910828,,, +,,,,72,2025-11-01 13:07:02.549834,,,,,,,0.21199890971183777,0.0032811176497489214,0.21199890971183777 +0.223603293299675,0.0022042153868824244,0.223603293299675,9099.0,73,2025-11-01 13:29:14.861842,,,,,,,,, +0.2442464530467987,0.002971305511891842,0.2442464530467987,9149.0,73,2025-11-01 13:51:20.291057,,,,,,,,, +,,,,73,2025-11-01 14:02:28.339801,0.20131665468215942,0.0031662345863878727,0.20131665468215942,0.19662335515022278,0.0025372703094035387,0.19662335515022278,,, +,,,,73,2025-11-01 14:03:08.284584,,,,,,,0.2126465141773224,0.003274420974776149,0.2126465141773224 +0.2335752695798874,0.0018764142878353596,0.2335752695798874,9199.0,74,2025-11-01 14:17:07.194597,,,,,,,,, diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/tf/events.out.tfevents.1761954553.node-0.2106.0 b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/tf/events.out.tfevents.1761954553.node-0.2106.0 new file mode 100644 index 0000000000000000000000000000000000000000..a87377cce415633a7f0a72b2becc8259930c3dd8 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_1/tf/events.out.tfevents.1761954553.node-0.2106.0 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bfde748bbd7f3d75a794e1dbcfa845ace1ea4b552f295ca34c9b90e987cab449 +size 18934 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/meta.experiment b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/meta.experiment new file mode 100644 index 0000000000000000000000000000000000000000..effd2a2b0a6c9695a718c07be0f65f82cd7036f5 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/meta.experiment @@ -0,0 +1 @@ +{"name": "testtube", "version": 2, "tags_path": "./logs/full_ct_2d_with_body_mask2/testtube/version_2/meta_tags.csv", "metrics_path": "./logs/full_ct_2d_with_body_mask2/testtube/version_2/metrics.csv", "autosave": false, "description": null, "created_at": "2025-12-11 20:44:30.894969", "exp_hash": "testtube_v2"} \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/meta_tags.csv b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/meta_tags.csv new file mode 100644 index 0000000000000000000000000000000000000000..942a47139f86778b1f3e5ad923104dd038b1ce88 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/meta_tags.csv @@ -0,0 +1 @@ +key,value diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/metrics.csv b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/metrics.csv new file mode 100644 index 0000000000000000000000000000000000000000..8b137891791fe96927ad78e64b0aad7bded08bdc --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/metrics.csv @@ -0,0 +1 @@ + diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/tf/events.out.tfevents.1765485873.node-0.2069.0 b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/tf/events.out.tfevents.1765485873.node-0.2069.0 new file mode 100644 index 0000000000000000000000000000000000000000..cdfd1e5773cb786ef2cf06d5a72110fe5fc3c730 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_2/tf/events.out.tfevents.1765485873.node-0.2069.0 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9944538160400aa0f28de20fd90fd78c7bfffd23c727f8a7497178018edc64f7 +size 88 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/meta.experiment b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/meta.experiment new file mode 100644 index 0000000000000000000000000000000000000000..6670f29ca3b67e75a61e0d2d10d1b047e14bcc6a --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/meta.experiment @@ -0,0 +1 @@ +{"name": "testtube", "version": 3, "tags_path": "./logs/full_ct_2d_with_body_mask2/testtube/version_3/meta_tags.csv", "metrics_path": "./logs/full_ct_2d_with_body_mask2/testtube/version_3/metrics.csv", "autosave": false, "description": null, "created_at": "2025-12-11 21:50:05.263192", "exp_hash": "testtube_v3"} \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/meta_tags.csv b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/meta_tags.csv new file mode 100644 index 0000000000000000000000000000000000000000..942a47139f86778b1f3e5ad923104dd038b1ce88 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/meta_tags.csv @@ -0,0 +1 @@ +key,value diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/metrics.csv b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/metrics.csv new file mode 100644 index 0000000000000000000000000000000000000000..53f0ba74c9a67a90af1e2d897d669a668e68e822 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/metrics.csv @@ -0,0 +1,1155 @@ +train/loss_simple_step,train/loss_vlb_step,train/loss_step,global_step,epoch,created_at,val/loss_simple,val/loss_vlb,val/loss,val/loss_simple_ema,val/loss_vlb_ema,val/loss_ema,train/loss_simple_epoch,train/loss_vlb_epoch,train/loss_epoch +0.14915211498737335,0.0013472875580191612,0.14915211498737335,9199.0,75,2025-12-11 21:59:05.169036,,,,,,,,, +0.20715951919555664,0.005803776904940605,0.20715951919555664,9249.0,75,2025-12-11 22:12:27.279429,,,,,,,,, +0.2342817485332489,0.004700809717178345,0.2342817485332489,9299.0,75,2025-12-11 22:28:14.001072,,,,,,,,, +,,,,75,2025-12-11 22:32:20.751305,0.20025181770324707,0.003180901287123561,0.20025181770324707,0.19995863735675812,0.0027487287297844887,0.19995863735675812,,, +,,,,75,2025-12-11 22:33:01.964924,,,,,,,0.21483054757118225,0.003265095641836524,0.21483054757118225 +0.20661796629428864,0.0016429716488346457,0.20661796629428864,9349.0,76,2025-12-11 22:51:14.561928,,,,,,,,, +0.1617777794599533,0.0010795173002406955,0.1617777794599533,9399.0,76,2025-12-11 23:03:51.179333,,,,,,,,, +,,,,76,2025-12-11 23:11:53.116467,0.19637171924114227,0.0031131186988204718,0.19637171924114227,0.19387680292129517,0.003704816335812211,0.19387680292129517,,, +,,,,76,2025-12-11 23:12:34.773836,,,,,,,0.21754227578639984,0.003319523297250271,0.21754227578639984 +0.18350301682949066,0.0024981950409710407,0.18350301682949066,9449.0,77,2025-12-11 23:22:41.130309,,,,,,,,, +0.22114711999893188,0.0028407960198819637,0.22114711999893188,9499.0,77,2025-12-11 23:35:14.059361,,,,,,,,, +,,,,77,2025-12-11 23:49:07.656241,0.20183487236499786,0.0023496095091104507,0.20183487236499786,0.19447417557239532,0.0029730102978646755,0.19447417557239532,,, +,,,,77,2025-12-11 23:49:49.314294,,,,,,,0.2132350504398346,0.0030718862544745207,0.2132350504398346 +0.22357846796512604,0.00670898100361228,0.22357846796512604,9549.0,78,2025-12-11 23:52:25.114528,,,,,,,,, +0.2516700327396393,0.012325514107942581,0.2516700327396393,9599.0,78,2025-12-12 00:06:36.049782,,,,,,,,, +0.21127226948738098,0.003272938309237361,0.21127226948738098,9649.0,78,2025-12-12 00:25:27.218084,,,,,,,,, +,,,,78,2025-12-12 00:32:58.422095,0.1966525912284851,0.0030156325083225965,0.1966525912284851,0.19965016841888428,0.002955154748633504,0.19965016841888428,,, +,,,,78,2025-12-12 00:33:39.553047,,,,,,,0.21422061324119568,0.0033990908414125443,0.21422061324119568 +0.1948951631784439,0.00154829490929842,0.1948951631784439,9699.0,79,2025-12-12 00:45:12.943447,,,,,,,,, +0.26043739914894104,0.00663595600053668,0.26043739914894104,9749.0,79,2025-12-12 01:01:12.188483,,,,,,,,, +,,,,79,2025-12-12 01:14:38.611244,0.19336019456386566,0.0033614225685596466,0.19336019456386566,0.19731009006500244,0.003358697285875678,0.19731009006500244,,, +,,,,79,2025-12-12 01:15:22.713372,,,,,,,0.2136542797088623,0.003131414530798793,0.2136542797088623 +0.26906663179397583,0.0061586759984493256,0.26906663179397583,9799.0,80,2025-12-12 01:21:05.372285,,,,,,,,, +0.1961517035961151,0.001191788469441235,0.1961517035961151,9849.0,80,2025-12-12 01:34:40.242294,,,,,,,,, +0.2342391014099121,0.005888995248824358,0.2342391014099121,9899.0,80,2025-12-12 01:50:18.160808,,,,,,,,, +,,,,80,2025-12-12 01:58:28.760892,0.19004416465759277,0.00258693122304976,0.19004416465759277,0.1890568733215332,0.004097824916243553,0.1890568733215332,,, +,,,,80,2025-12-12 01:59:12.606711,,,,,,,0.2153792828321457,0.003361398819833994,0.2153792828321457 +0.20689773559570312,0.003979964647442102,0.20689773559570312,9949.0,81,2025-12-12 02:09:57.186862,,,,,,,,, +0.22295133769512177,0.004203625023365021,0.22295133769512177,9999.0,81,2025-12-12 02:24:09.122904,,,,,,,,, +,,,,81,2025-12-12 02:38:36.244804,0.1909826397895813,0.0030700804200023413,0.1909826397895813,0.18513816595077515,0.003034281777217984,0.18513816595077515,,, +,,,,81,2025-12-12 02:39:19.575388,,,,,,,0.2170076221227646,0.0031740895938128233,0.2170076221227646 +0.24744659662246704,0.0016126279952004552,0.24744659662246704,10049.0,82,2025-12-12 02:44:28.421852,,,,,,,,, +0.18222488462924957,0.002596472855657339,0.18222488462924957,10099.0,82,2025-12-12 02:54:21.105791,,,,,,,,, +0.22754445672035217,0.005961373448371887,0.22754445672035217,10149.0,82,2025-12-12 03:05:46.228757,,,,,,,,, +,,,,82,2025-12-12 03:16:29.274742,0.20467722415924072,0.003249090164899826,0.20467722415924072,0.20107778906822205,0.0028579605277627707,0.20107778906822205,,, +,,,,82,2025-12-12 03:16:53.079269,,,,,,,0.21404477953910828,0.0032218617852777243,0.21404477953910828 +0.21761460602283478,0.003961848095059395,0.21761460602283478,10199.0,83,2025-12-12 03:25:12.683686,,,,,,,,, +0.23437795042991638,0.0026058186776936054,0.23437795042991638,10249.0,83,2025-12-12 03:36:04.439552,,,,,,,,, +,,,,83,2025-12-12 03:52:42.008169,0.18967171013355255,0.0034202798269689083,0.18967171013355255,0.19652459025382996,0.002223290503025055,0.19652459025382996,,, +,,,,83,2025-12-12 03:53:05.131548,,,,,,,0.21285521984100342,0.003259174060076475,0.21285521984100342 +0.19085533916950226,0.0012468273052945733,0.19085533916950226,10299.0,84,2025-12-12 04:03:32.535452,,,,,,,,, +0.21291452646255493,0.0014697364531457424,0.21291452646255493,10349.0,84,2025-12-12 04:12:47.585866,,,,,,,,, +0.2151278555393219,0.003228934947401285,0.2151278555393219,10399.0,84,2025-12-12 04:27:51.309078,,,,,,,,, +,,,,84,2025-12-12 04:34:42.501410,0.18278592824935913,0.0028544028755277395,0.18278592824935913,0.19322383403778076,0.002995193935930729,0.19322383403778076,,, +,,,,84,2025-12-12 04:35:28.415252,,,,,,,0.21078214049339294,0.002930614398792386,0.21078214049339294 +0.208448126912117,0.0012576557928696275,0.208448126912117,10449.0,85,2025-12-12 04:45:52.511841,,,,,,,,, +0.19752444326877594,0.0018394661601632833,0.19752444326877594,10499.0,85,2025-12-12 04:58:20.326812,,,,,,,,, +,,,,85,2025-12-12 05:13:10.775642,0.18713882565498352,0.00170434161555022,0.18713882565498352,0.19655445218086243,0.0030149996746331453,0.19655445218086243,,, +,,,,85,2025-12-12 05:13:33.969107,,,,,,,0.208177849650383,0.00303513091057539,0.208177849650383 +0.16027326881885529,0.0018350210739299655,0.16027326881885529,10549.0,86,2025-12-12 05:18:20.362220,,,,,,,,, +0.24760879576206207,0.001815914991311729,0.24760879576206207,10599.0,86,2025-12-12 05:32:41.448932,,,,,,,,, +0.20129115879535675,0.002938847756013274,0.20129115879535675,10649.0,86,2025-12-12 05:46:01.026010,,,,,,,,, +,,,,86,2025-12-12 05:52:48.784606,0.19269227981567383,0.0021264958195388317,0.19269227981567383,0.1971500664949417,0.0029645308386534452,0.1971500664949417,,, +,,,,86,2025-12-12 05:53:12.155320,,,,,,,0.2092648297548294,0.002863414352759719,0.2092648297548294 +0.1598220020532608,0.004399426281452179,0.1598220020532608,10699.0,87,2025-12-12 06:05:24.918779,,,,,,,,, +0.21266478300094604,0.003136760089546442,0.21266478300094604,10749.0,87,2025-12-12 06:21:58.890915,,,,,,,,, +,,,,87,2025-12-12 06:37:58.396308,0.18921208381652832,0.003731902688741684,0.18921208381652832,0.19599191844463348,0.0032396928872913122,0.19599191844463348,,, +,,,,87,2025-12-12 06:38:21.840035,,,,,,,0.21354730427265167,0.0033477756660431623,0.21354730427265167 +0.24152615666389465,0.0030458825640380383,0.24152615666389465,10799.0,88,2025-12-12 06:44:12.412833,,,,,,,,, +0.18110577762126923,0.001960690598934889,0.18110577762126923,10849.0,88,2025-12-12 06:57:05.889171,,,,,,,,, +0.2544110119342804,0.006359439808875322,0.2544110119342804,10899.0,88,2025-12-12 07:08:18.421283,,,,,,,,, +,,,,88,2025-12-12 07:19:56.953334,0.19448408484458923,0.0026776641607284546,0.19448408484458923,0.19465821981430054,0.0032826075330376625,0.19465821981430054,,, +,,,,88,2025-12-12 07:20:20.129126,,,,,,,0.21474015712738037,0.003311052219942212,0.21474015712738037 +0.22152115404605865,0.010662391781806946,0.22152115404605865,10949.0,89,2025-12-12 07:33:50.204142,,,,,,,,, +0.18408764898777008,0.0020414814352989197,0.18408764898777008,10999.0,89,2025-12-12 07:46:18.852137,,,,,,,,, +,,,,89,2025-12-12 07:59:30.403517,0.19848759472370148,0.0031447766814380884,0.19848759472370148,0.19386735558509827,0.0025198080111294985,0.19386735558509827,,, +,,,,89,2025-12-12 07:59:55.241529,,,,,,,0.21182361245155334,0.0032167506869882345,0.21182361245155334 +0.21015526354312897,0.0032799646724015474,0.21015526354312897,11049.0,90,2025-12-12 08:08:19.331302,,,,,,,,, +0.24825197458267212,0.002562514040619135,0.24825197458267212,11099.0,90,2025-12-12 08:20:31.814183,,,,,,,,, +0.23667781054973602,0.006114631425589323,0.23667781054973602,11149.0,90,2025-12-12 08:31:42.626567,,,,,,,,, +,,,,90,2025-12-12 08:38:28.952326,0.188440203666687,0.0021121776662766933,0.188440203666687,0.19691318273544312,0.002352542243897915,0.19691318273544312,,, +,,,,90,2025-12-12 08:38:54.654873,,,,,,,0.211579829454422,0.003336614929139614,0.211579829454422 +0.2873271703720093,0.007237555459141731,0.2873271703720093,11199.0,91,2025-12-12 08:50:19.026278,,,,,,,,, +0.1886076033115387,0.0035629034973680973,0.1886076033115387,11249.0,91,2025-12-12 09:08:05.688055,,,,,,,,, +,,,,91,2025-12-12 09:20:18.464729,0.2081824541091919,0.004163067787885666,0.2081824541091919,0.19381548464298248,0.002563128247857094,0.19381548464298248,,, +,,,,91,2025-12-12 09:20:41.454362,,,,,,,0.20828455686569214,0.00315662007778883,0.20828455686569214 +0.20845741033554077,0.0016812040703371167,0.20845741033554077,11299.0,92,2025-12-12 09:27:10.599713,,,,,,,,, +0.2356138527393341,0.0017496931832283735,0.2356138527393341,11349.0,92,2025-12-12 09:39:44.907342,,,,,,,,, +0.19643104076385498,0.0025110975839197636,0.19643104076385498,11399.0,92,2025-12-12 09:50:57.661285,,,,,,,,, +,,,,92,2025-12-12 09:56:23.477880,0.18757161498069763,0.0026203938759863377,0.18757161498069763,0.1943749189376831,0.002875677077099681,0.1943749189376831,,, +,,,,92,2025-12-12 09:56:47.471781,,,,,,,0.21305254101753235,0.0029891212470829487,0.21305254101753235 +0.18571625649929047,0.004724969621747732,0.18571625649929047,11449.0,93,2025-12-12 10:11:01.056352,,,,,,,,, +0.23140232264995575,0.002478665905073285,0.23140232264995575,11499.0,93,2025-12-12 10:25:06.984871,,,,,,,,, +,,,,93,2025-12-12 10:35:54.914376,0.18565040826797485,0.0024980560410767794,0.18565040826797485,0.19764086604118347,0.00315352831967175,0.19764086604118347,,, +,,,,93,2025-12-12 10:36:18.152512,,,,,,,0.2121652215719223,0.0032446107361465693,0.2121652215719223 +0.20776647329330444,0.0023922366090118885,0.20776647329330444,11549.0,94,2025-12-12 10:44:26.267737,,,,,,,,, +0.1974518746137619,0.001986215589568019,0.1974518746137619,11599.0,94,2025-12-12 10:58:10.305542,,,,,,,,, +0.19799157977104187,0.0016093978192657232,0.19799157977104187,11649.0,94,2025-12-12 11:08:47.633023,,,,,,,,, +,,,,94,2025-12-12 11:13:43.396511,0.19760477542877197,0.003111376194283366,0.19760477542877197,0.1892394870519638,0.0034588552080094814,0.1892394870519638,,, +,,,,94,2025-12-12 11:14:27.432386,,,,,,,0.2140541970729828,0.0034884759224951267,0.2140541970729828 +0.26194941997528076,0.003769511356949806,0.26194941997528076,11699.0,95,2025-12-12 11:28:55.120926,,,,,,,,, +0.20761257410049438,0.002903405809774995,0.20761257410049438,11749.0,95,2025-12-12 11:43:01.649449,,,,,,,,, +,,,,95,2025-12-12 11:53:21.324694,0.1947278529405594,0.003203820204362273,0.1947278529405594,0.19914911687374115,0.001987694762647152,0.19914911687374115,,, +,,,,95,2025-12-12 11:53:45.026312,,,,,,,0.2086726576089859,0.0032490924932062626,0.2086726576089859 +0.17984989285469055,0.0029694202821701765,0.17984989285469055,11799.0,96,2025-12-12 12:00:44.831664,,,,,,,,, +0.19478553533554077,0.005223124288022518,0.19478553533554077,11849.0,96,2025-12-12 12:15:18.903362,,,,,,,,, +0.2125595062971115,0.006968900561332703,0.2125595062971115,11899.0,96,2025-12-12 12:25:57.092620,,,,,,,,, +,,,,96,2025-12-12 12:33:42.446121,0.20219391584396362,0.004193316679447889,0.20219391584396362,0.19338633120059967,0.002207711571827531,0.19338633120059967,,, +,,,,96,2025-12-12 12:34:06.358643,,,,,,,0.2080010324716568,0.003289623884484172,0.2080010324716568 +0.17825381457805634,0.0010953033342957497,0.17825381457805634,11949.0,97,2025-12-12 12:49:58.591372,,,,,,,,, +0.23918606340885162,0.0017230352386832237,0.23918606340885162,11999.0,97,2025-12-12 13:02:36.022001,,,,,,,,, +,,,,97,2025-12-12 13:12:12.337020,0.19402849674224854,0.0036673061549663544,0.19402849674224854,0.1921621859073639,0.0037490492686629295,0.1921621859073639,,, +,,,,97,2025-12-12 13:12:34.703454,,,,,,,0.20786182582378387,0.0029700477607548237,0.20786182582378387 +0.22021712362766266,0.0020936017390340567,0.22021712362766266,12049.0,98,2025-12-12 13:24:37.422344,,,,,,,,, +0.22014513611793518,0.0036847288720309734,0.22014513611793518,12099.0,98,2025-12-12 13:38:35.161867,,,,,,,,, +0.16601133346557617,0.0024280333891510963,0.16601133346557617,12149.0,98,2025-12-12 13:53:51.474005,,,,,,,,, +,,,,98,2025-12-12 13:58:12.185436,0.1817232370376587,0.0021073073148727417,0.1817232370376587,0.194093257188797,0.002673766575753689,0.194093257188797,,, +,,,,98,2025-12-12 13:58:35.591779,,,,,,,0.21022829413414001,0.0032337165903300047,0.21022829413414001 +0.1954358071088791,0.0035670085344463587,0.1954358071088791,12199.0,99,2025-12-12 14:17:28.871882,,,,,,,,, +0.19359177350997925,0.004970996640622616,0.19359177350997925,12249.0,99,2025-12-12 14:27:38.392292,,,,,,,,, +,,,,99,2025-12-12 14:36:17.799551,0.19162069261074066,0.0030196320731192827,0.19162069261074066,0.19214600324630737,0.0019563455134630203,0.19214600324630737,,, +,,,,99,2025-12-12 14:36:40.334043,,,,,,,0.20447960495948792,0.0026236444246023893,0.20447960495948792 +0.18105833232402802,0.0015051591908559203,0.18105833232402802,12299.0,100,2025-12-12 14:45:11.723641,,,,,,,,, +0.2306917905807495,0.002302119042724371,0.2306917905807495,12349.0,100,2025-12-12 14:59:34.270155,,,,,,,,, +0.20200973749160767,0.0038564521819353104,0.20200973749160767,12399.0,100,2025-12-12 15:12:32.488420,,,,,,,,, +,,,,100,2025-12-12 15:16:37.512739,0.19685529172420502,0.0034311828203499317,0.19685529172420502,0.1960335522890091,0.0023243939504027367,0.1960335522890091,,, +,,,,100,2025-12-12 15:17:00.710437,,,,,,,0.2098497897386551,0.002887510461732745,0.2098497897386551 +0.1976003795862198,0.0018983001355081797,0.1976003795862198,12449.0,101,2025-12-12 15:29:55.387404,,,,,,,,, +0.17915038764476776,0.0015258147614076734,0.17915038764476776,12499.0,101,2025-12-12 15:42:07.809506,,,,,,,,, +,,,,101,2025-12-12 15:53:47.496256,0.19488823413848877,0.0018287347629666328,0.19488823413848877,0.1926816701889038,0.00357631198130548,0.1926816701889038,,, +,,,,101,2025-12-12 15:54:10.469552,,,,,,,0.2104259580373764,0.0031717277597635984,0.2104259580373764 +0.18399129807949066,0.004490786697715521,0.18399129807949066,12549.0,102,2025-12-12 16:04:22.883505,,,,,,,,, +0.23373425006866455,0.001416694954968989,0.23373425006866455,12599.0,102,2025-12-12 16:16:55.464993,,,,,,,,, +,,,,102,2025-12-12 16:34:53.558749,0.20378388464450836,0.004255295731127262,0.20378388464450836,0.18346258997917175,0.003489105962216854,0.18346258997917175,,, +,,,,102,2025-12-12 16:35:38.503429,,,,,,,0.20762354135513306,0.0030355544295161963,0.20762354135513306 +0.21204020082950592,0.0019014622084796429,0.21204020082950592,12649.0,103,2025-12-12 16:37:55.976476,,,,,,,,, +0.21928374469280243,0.0019280612468719482,0.21928374469280243,12699.0,103,2025-12-12 16:52:46.109066,,,,,,,,, +0.23586943745613098,0.0015401964774355292,0.23586943745613098,12749.0,103,2025-12-12 17:07:43.246897,,,,,,,,, +,,,,103,2025-12-12 17:22:21.299061,0.18439070880413055,0.0034736027009785175,0.18439070880413055,0.19013255834579468,0.002959535224363208,0.19013255834579468,,, +,,,,103,2025-12-12 17:22:44.464730,,,,,,,0.2095859944820404,0.0030432301573455334,0.2095859944820404 +0.2131330370903015,0.001965276198461652,0.2131330370903015,12799.0,104,2025-12-12 17:32:09.208010,,,,,,,,, +0.20780175924301147,0.003335619578137994,0.20780175924301147,12849.0,104,2025-12-12 17:44:27.658784,,,,,,,,, +,,,,104,2025-12-12 18:00:40.191389,0.19784393906593323,0.003011156804859638,0.19784393906593323,0.19364014267921448,0.003402628004550934,0.19364014267921448,,, +,,,,104,2025-12-12 18:01:03.271159,,,,,,,0.20877374708652496,0.002792564919218421,0.20877374708652496 +0.2097260057926178,0.00507203908637166,0.2097260057926178,12899.0,105,2025-12-12 18:03:37.765533,,,,,,,,, +0.1884150356054306,0.005102515686303377,0.1884150356054306,12949.0,105,2025-12-12 18:21:49.762540,,,,,,,,, +0.2198498547077179,0.005215873010456562,0.2198498547077179,12999.0,105,2025-12-12 18:34:04.975268,,,,,,,,, +,,,,105,2025-12-12 18:42:00.150520,0.19400793313980103,0.0030836048536002636,0.19400793313980103,0.19311468303203583,0.0019448664970695972,0.19311468303203583,,, +,,,,105,2025-12-12 18:42:23.812590,,,,,,,0.20782555639743805,0.0030485426541417837,0.20782555639743805 +0.2551656663417816,0.005593480542302132,0.2551656663417816,13049.0,106,2025-12-12 18:53:29.339183,,,,,,,,, +0.20576444268226624,0.0046198503114283085,0.20576444268226624,13099.0,106,2025-12-12 19:11:48.757909,,,,,,,,, +,,,,106,2025-12-12 19:24:11.934335,0.1947270631790161,0.002373219933360815,0.1947270631790161,0.1960655152797699,0.0019478340400382876,0.1960655152797699,,, +,,,,106,2025-12-12 19:24:34.415427,,,,,,,0.20508550107479095,0.0032610874623060226,0.20508550107479095 +0.1918020099401474,0.001485395710915327,0.1918020099401474,13149.0,107,2025-12-12 19:27:25.552223,,,,,,,,, +0.2195984274148941,0.009934398345649242,0.2195984274148941,13199.0,107,2025-12-12 19:39:13.441635,,,,,,,,, +0.2146318107843399,0.006799294147640467,0.2146318107843399,13249.0,107,2025-12-12 19:50:03.283283,,,,,,,,, +,,,,107,2025-12-12 19:58:05.811589,0.1950482428073883,0.00267709419131279,0.1950482428073883,0.1879342943429947,0.0033623983617872,0.1879342943429947,,, +,,,,107,2025-12-12 19:58:52.259420,,,,,,,0.20989356935024261,0.0032016539480537176,0.20989356935024261 +0.1706613302230835,0.0036120552103966475,0.1706613302230835,13299.0,108,2025-12-12 20:10:43.292541,,,,,,,,, +0.1780886948108673,0.0014993372606113553,0.1780886948108673,13349.0,108,2025-12-12 20:31:05.506649,,,,,,,,, +,,,,108,2025-12-12 20:43:25.111048,0.18546512722969055,0.001972048543393612,0.18546512722969055,0.195102259516716,0.0036206406075507402,0.195102259516716,,, +,,,,108,2025-12-12 20:43:47.770332,,,,,,,0.2099449336528778,0.003410800825804472,0.2099449336528778 +0.21675321459770203,0.0016939436318352818,0.21675321459770203,13399.0,109,2025-12-12 20:49:51.125779,,,,,,,,, +0.1912328153848648,0.0034970019478350878,0.1912328153848648,13449.0,109,2025-12-12 21:02:57.547002,,,,,,,,, +0.19078931212425232,0.002170519670471549,0.19078931212425232,13499.0,109,2025-12-12 21:16:55.244458,,,,,,,,, +,,,,109,2025-12-12 21:26:15.608670,0.1879945993423462,0.0027085414621979,0.1879945993423462,0.19183871150016785,0.0022857452277094126,0.19183871150016785,,, +,,,,109,2025-12-12 21:26:39.503590,,,,,,,0.21360211074352264,0.003099270863458514,0.21360211074352264 +0.20482788980007172,0.0014021306997165084,0.20482788980007172,13549.0,110,2025-12-12 21:36:58.889328,,,,,,,,, +0.17406915128231049,0.0011887747095897794,0.17406915128231049,13599.0,110,2025-12-12 21:51:17.621151,,,,,,,,, +,,,,110,2025-12-12 22:06:24.352382,0.20111815631389618,0.0030794856138527393,0.20111815631389618,0.19190429151058197,0.0033228914253413677,0.19190429151058197,,, +,,,,110,2025-12-12 22:06:47.793411,,,,,,,0.21093715727329254,0.003018953138962388,0.21093715727329254 +0.20926879346370697,0.0016213060589507222,0.20926879346370697,13649.0,111,2025-12-12 22:17:11.324873,,,,,,,,, +0.23838281631469727,0.0014748176326975226,0.23838281631469727,13699.0,111,2025-12-12 22:25:39.430610,,,,,,,,, +0.15621954202651978,0.002152529079467058,0.15621954202651978,13749.0,111,2025-12-12 22:36:24.384185,,,,,,,,, +,,,,111,2025-12-12 22:44:07.404581,0.18804721534252167,0.0024277062620967627,0.18804721534252167,0.18887802958488464,0.003176940605044365,0.18887802958488464,,, +,,,,111,2025-12-12 22:44:31.092862,,,,,,,0.2076963633298874,0.0032215744722634554,0.2076963633298874 +0.18253977596759796,0.0014084524009376764,0.18253977596759796,13799.0,112,2025-12-12 22:57:38.624656,,,,,,,,, +0.173243448138237,0.005463361274451017,0.173243448138237,13849.0,112,2025-12-12 23:11:05.511116,,,,,,,,, +,,,,112,2025-12-12 23:28:34.963409,0.19160062074661255,0.0024910280480980873,0.19160062074661255,0.19018638134002686,0.003733585588634014,0.19018638134002686,,, +,,,,112,2025-12-12 23:28:59.277517,,,,,,,0.20232877135276794,0.0033193703275173903,0.20232877135276794 +0.185489684343338,0.001260058139450848,0.185489684343338,13899.0,113,2025-12-12 23:35:05.294508,,,,,,,,, +0.1994747519493103,0.0010803567711263895,0.1994747519493103,13949.0,113,2025-12-12 23:47:38.662771,,,,,,,,, +0.22356027364730835,0.002563812304288149,0.22356027364730835,13999.0,113,2025-12-12 23:59:17.240133,,,,,,,,, +,,,,113,2025-12-13 00:07:46.812115,0.20713840425014496,0.002961745485663414,0.20713840425014496,0.1945337951183319,0.0031192898750305176,0.1945337951183319,,, +,,,,113,2025-12-13 00:08:10.449085,,,,,,,0.2065124809741974,0.0031928906682878733,0.2065124809741974 +0.23276913166046143,0.006394263822585344,0.23276913166046143,14049.0,114,2025-12-13 00:19:37.425117,,,,,,,,, +0.23554261028766632,0.002867315663024783,0.23554261028766632,14099.0,114,2025-12-13 00:31:28.363013,,,,,,,,, +,,,,114,2025-12-13 00:42:29.433088,0.18677634000778198,0.0032521227840334177,0.18677634000778198,0.1939181685447693,0.002902309875935316,0.1939181685447693,,, +,,,,114,2025-12-13 00:42:54.075252,,,,,,,0.2048271745443344,0.0030727689154446125,0.2048271745443344 +0.20103709399700165,0.001359549700282514,0.20103709399700165,14149.0,115,2025-12-13 00:48:55.279855,,,,,,,,, +0.19937454164028168,0.008906775154173374,0.19937454164028168,14199.0,115,2025-12-13 01:04:19.667035,,,,,,,,, +0.24131269752979279,0.0023297234438359737,0.24131269752979279,14249.0,115,2025-12-13 01:16:28.902244,,,,,,,,, +,,,,115,2025-12-13 01:22:14.083433,0.17969009280204773,0.002606638940051198,0.17969009280204773,0.1994571089744568,0.003045763587579131,0.1994571089744568,,, +,,,,115,2025-12-13 01:22:37.557798,,,,,,,0.20942054688930511,0.002971068723127246,0.20942054688930511 +0.2465096116065979,0.01020241528749466,0.2465096116065979,14299.0,116,2025-12-13 01:33:27.961355,,,,,,,,, +0.22180767357349396,0.006664034444838762,0.22180767357349396,14349.0,116,2025-12-13 01:45:11.751685,,,,,,,,, +,,,,116,2025-12-13 01:56:40.135698,0.18146497011184692,0.002924778498709202,0.18146497011184692,0.20417967438697815,0.0028884445782750845,0.20417967438697815,,, +,,,,116,2025-12-13 01:57:03.672130,,,,,,,0.20882907509803772,0.0032431879080832005,0.20882907509803772 +0.18371784687042236,0.006901541259139776,0.18371784687042236,14399.0,117,2025-12-13 02:06:24.872594,,,,,,,,, +0.20194275677204132,0.0013735187239944935,0.20194275677204132,14449.0,117,2025-12-13 02:19:15.136408,,,,,,,,, +0.20787544548511505,0.007341769523918629,0.20787544548511505,14499.0,117,2025-12-13 02:29:28.595247,,,,,,,,, +,,,,117,2025-12-13 02:36:56.629312,0.18489450216293335,0.002406064420938492,0.18489450216293335,0.20009735226631165,0.0036936982069164515,0.20009735226631165,,, +,,,,117,2025-12-13 02:37:20.587114,,,,,,,0.20730960369110107,0.0031573567539453506,0.20730960369110107 +0.1860765665769577,0.0018485394539311528,0.1860765665769577,14549.0,118,2025-12-13 02:48:01.389730,,,,,,,,, +0.22180970013141632,0.0023936377838253975,0.22180970013141632,14599.0,118,2025-12-13 03:00:02.394173,,,,,,,,, +,,,,118,2025-12-13 03:10:24.112049,0.19546929001808167,0.002994125708937645,0.19546929001808167,0.19742326438426971,0.002355085453018546,0.19742326438426971,,, +,,,,118,2025-12-13 03:10:47.415689,,,,,,,0.20632222294807434,0.003328481689095497,0.20632222294807434 +0.21472980082035065,0.008340098895132542,0.21472980082035065,14649.0,119,2025-12-13 03:18:32.159221,,,,,,,,, +0.23530219495296478,0.009962365962564945,0.23530219495296478,14699.0,119,2025-12-13 03:33:27.372709,,,,,,,,, +0.20269045233726501,0.0022037914022803307,0.20269045233726501,14749.0,119,2025-12-13 03:48:47.288908,,,,,,,,, +,,,,119,2025-12-13 03:53:50.879591,0.18173597753047943,0.002191110048443079,0.18173597753047943,0.196058452129364,0.003579817246645689,0.196058452129364,,, +,,,,119,2025-12-13 03:54:14.811761,,,,,,,0.2107665091753006,0.00315764918923378,0.2107665091753006 +0.20172260701656342,0.0016792876413092017,0.20172260701656342,14799.0,120,2025-12-13 04:07:54.140832,,,,,,,,, +0.16991935670375824,0.0011571093928068876,0.16991935670375824,14849.0,120,2025-12-13 04:17:37.889129,,,,,,,,, +,,,,120,2025-12-13 04:30:23.008172,0.18612849712371826,0.0022677357774227858,0.18612849712371826,0.17808352410793304,0.0017960589611902833,0.17808352410793304,,, +,,,,120,2025-12-13 04:31:09.175595,,,,,,,0.20661352574825287,0.002976383315399289,0.20661352574825287 +0.17859715223312378,0.0014713220298290253,0.17859715223312378,14899.0,121,2025-12-13 04:43:29.907706,,,,,,,,, +0.1662418097257614,0.0012485119514167309,0.1662418097257614,14949.0,121,2025-12-13 04:57:07.262568,,,,,,,,, +0.17450609803199768,0.0015479115536436439,0.17450609803199768,14999.0,121,2025-12-13 05:07:11.073599,,,,,,,,, +,,,,121,2025-12-13 05:12:18.487396,0.19905513525009155,0.0032440286595374346,0.19905513525009155,0.1960793435573578,0.0024416306987404823,0.1960793435573578,,, +,,,,121,2025-12-13 05:12:42.367415,,,,,,,0.20591877400875092,0.002851600991562009,0.20591877400875092 +0.191749706864357,0.0018218914046883583,0.191749706864357,15049.0,122,2025-12-13 05:26:58.959470,,,,,,,,, +0.16077759861946106,0.001295587164349854,0.16077759861946106,15099.0,122,2025-12-13 05:35:15.410040,,,,,,,,, +,,,,122,2025-12-13 05:45:25.401910,0.2014632225036621,0.003137728199362755,0.2014632225036621,0.1738012433052063,0.0018159893807023764,0.1738012433052063,,, +,,,,122,2025-12-13 05:46:14.420935,,,,,,,0.20513185858726501,0.003092549741268158,0.20513185858726501 +0.2176397144794464,0.003276142058894038,0.2176397144794464,15149.0,123,2025-12-13 05:54:32.446341,,,,,,,,, +0.19087733328342438,0.001919238711707294,0.19087733328342438,15199.0,123,2025-12-13 06:07:45.703631,,,,,,,,, +0.21787278354167938,0.0033659925684332848,0.21787278354167938,15249.0,123,2025-12-13 06:17:01.337661,,,,,,,,, +,,,,123,2025-12-13 06:21:32.693257,0.19451212882995605,0.0021568331867456436,0.19451212882995605,0.19162946939468384,0.0036737711634486914,0.19162946939468384,,, +,,,,123,2025-12-13 06:21:56.906497,,,,,,,0.20748649537563324,0.002919102320447564,0.20748649537563324 +0.20137067139148712,0.0018256001640111208,0.20137067139148712,15299.0,124,2025-12-13 06:35:00.859852,,,,,,,,, +0.16464199125766754,0.0061327507719397545,0.16464199125766754,15349.0,124,2025-12-13 06:44:21.211544,,,,,,,,, +,,,,124,2025-12-13 06:57:12.291666,0.1937229186296463,0.002649920294061303,0.1937229186296463,0.18568986654281616,0.002446656348183751,0.18568986654281616,,, +,,,,124,2025-12-13 06:57:36.097811,,,,,,,0.20640966296195984,0.0030540849547833204,0.20640966296195984 +0.22327853739261627,0.006944504100829363,0.22327853739261627,15399.0,125,2025-12-13 07:05:54.289229,,,,,,,,, +0.12416999787092209,0.001497172866947949,0.12416999787092209,15449.0,125,2025-12-13 07:21:19.278589,,,,,,,,, +0.20217746496200562,0.001658508088439703,0.20217746496200562,15499.0,125,2025-12-13 07:33:12.953576,,,,,,,,, +,,,,125,2025-12-13 07:37:18.059630,0.19404780864715576,0.0031580536160618067,0.19404780864715576,0.19863462448120117,0.0038429503329098225,0.19863462448120117,,, +,,,,125,2025-12-13 07:37:42.597858,,,,,,,0.20438258349895477,0.0031225206330418587,0.20438258349895477 +0.22112974524497986,0.0017548883333802223,0.22112974524497986,15549.0,126,2025-12-13 07:50:17.036584,,,,,,,,, +0.21588802337646484,0.001479986822232604,0.21588802337646484,15599.0,126,2025-12-13 08:04:33.263020,,,,,,,,, +,,,,126,2025-12-13 08:13:45.536683,0.19480837881565094,0.0027083884924650192,0.19480837881565094,0.20409217476844788,0.0036762505769729614,0.20409217476844788,,, +,,,,126,2025-12-13 08:14:09.595410,,,,,,,0.20612961053848267,0.003225162159651518,0.20612961053848267 +0.20532000064849854,0.0019201167160645127,0.20532000064849854,15649.0,127,2025-12-13 08:22:43.933599,,,,,,,,, +0.20732058584690094,0.001693626050837338,0.20732058584690094,15699.0,127,2025-12-13 08:35:46.367428,,,,,,,,, +,,,,127,2025-12-13 08:53:59.688594,0.17419099807739258,0.0021758449729532003,0.17419099807739258,0.19546811282634735,0.0020194868557155132,0.19546811282634735,,, +,,,,127,2025-12-13 08:54:23.864063,,,,,,,0.20915673673152924,0.0031099242623895407,0.20915673673152924 +0.21676227450370789,0.0016267676837742329,0.21676227450370789,15749.0,128,2025-12-13 08:58:37.136736,,,,,,,,, +0.20302677154541016,0.0013065958628430963,0.20302677154541016,15799.0,128,2025-12-13 09:09:26.003727,,,,,,,,, +0.23496675491333008,0.0013010365655645728,0.23496675491333008,15849.0,128,2025-12-13 09:18:37.300569,,,,,,,,, +,,,,128,2025-12-13 09:28:35.166191,0.196193128824234,0.0033747151028364897,0.196193128824234,0.18649500608444214,0.002579980529844761,0.18649500608444214,,, +,,,,128,2025-12-13 09:29:00.080470,,,,,,,0.2075926512479782,0.003020084463059902,0.2075926512479782 +0.1921606957912445,0.0021048227790743113,0.1921606957912445,15899.0,129,2025-12-13 09:41:58.422010,,,,,,,,, +0.18405325710773468,0.0016858665039762855,0.18405325710773468,15949.0,129,2025-12-13 09:58:15.699042,,,,,,,,, +,,,,129,2025-12-13 10:12:14.712101,0.19077737629413605,0.0025105022359639406,0.19077737629413605,0.18876969814300537,0.003049226012080908,0.18876969814300537,,, +,,,,129,2025-12-13 10:12:38.569337,,,,,,,0.20390470325946808,0.00329759088344872,0.20390470325946808 +0.17581695318222046,0.001802976825274527,0.17581695318222046,15999.0,130,2025-12-13 10:16:20.204727,,,,,,,,, +0.1917804330587387,0.002764076227322221,0.1917804330587387,16049.0,130,2025-12-13 10:32:26.794060,,,,,,,,, +0.2136046439409256,0.007173709571361542,0.2136046439409256,16099.0,130,2025-12-13 10:44:25.290992,,,,,,,,, +,,,,130,2025-12-13 10:55:06.595880,0.19460177421569824,0.0034116674214601517,0.19460177421569824,0.18824630975723267,0.002860638312995434,0.18824630975723267,,, +,,,,130,2025-12-13 10:55:29.892714,,,,,,,0.20745328068733215,0.003078080015257001,0.20745328068733215 +0.20385392010211945,0.0021764924749732018,0.20385392010211945,16149.0,131,2025-12-13 11:04:31.571313,,,,,,,,, +0.22873659431934357,0.005520596168935299,0.22873659431934357,16199.0,131,2025-12-13 11:17:55.824557,,,,,,,,, +,,,,131,2025-12-13 11:31:35.056903,0.18791140615940094,0.0032488107681274414,0.18791140615940094,0.19535517692565918,0.0033917652908712626,0.19535517692565918,,, +,,,,131,2025-12-13 11:31:59.429215,,,,,,,0.20753589272499084,0.0032250734511762857,0.20753589272499084 +0.20218586921691895,0.001255681156180799,0.20218586921691895,16249.0,132,2025-12-13 11:35:53.978613,,,,,,,,, +0.22068114578723907,0.0014661467866972089,0.22068114578723907,16299.0,132,2025-12-13 11:47:16.963397,,,,,,,,, +0.17897367477416992,0.004543696530163288,0.17897367477416992,16349.0,132,2025-12-13 11:57:25.538131,,,,,,,,, +,,,,132,2025-12-13 12:06:41.420211,0.18623098731040955,0.0033829514868557453,0.18623098731040955,0.1774539053440094,0.0017366952961310744,0.1774539053440094,,, +,,,,132,2025-12-13 12:07:24.800306,,,,,,,0.20825278759002686,0.00302325701341033,0.20825278759002686 +0.26490864157676697,0.0028725778684020042,0.26490864157676697,16399.0,133,2025-12-13 12:18:14.288704,,,,,,,,, +0.2137090265750885,0.0016834981506690383,0.2137090265750885,16449.0,133,2025-12-13 12:32:23.500475,,,,,,,,, +,,,,133,2025-12-13 12:44:35.279836,0.19017036259174347,0.0033815328497439623,0.19017036259174347,0.183820441365242,0.003089785110205412,0.183820441365242,,, +,,,,133,2025-12-13 12:44:59.447079,,,,,,,0.20744438469409943,0.0031939824111759663,0.20744438469409943 +0.20409291982650757,0.0049581993371248245,0.20409291982650757,16499.0,134,2025-12-13 12:51:18.266890,,,,,,,,, +0.21639518439769745,0.0060355644673109055,0.21639518439769745,16549.0,134,2025-12-13 13:01:27.349524,,,,,,,,, +0.18989072740077972,0.0027370138559490442,0.18989072740077972,16599.0,134,2025-12-13 13:13:57.888713,,,,,,,,, +,,,,134,2025-12-13 13:21:03.522182,0.18853235244750977,0.00489760423079133,0.18853235244750977,0.18405738472938538,0.0018997370498254895,0.18405738472938538,,, +,,,,134,2025-12-13 13:21:27.742266,,,,,,,0.20195023715496063,0.002979455515742302,0.20195023715496063 +0.1628510057926178,0.0011935385409742594,0.1628510057926178,16649.0,135,2025-12-13 13:32:34.086846,,,,,,,,, +0.1858977973461151,0.0013817287981510162,0.1858977973461151,16699.0,135,2025-12-13 13:46:24.689449,,,,,,,,, +,,,,135,2025-12-13 13:57:08.918820,0.1901337057352066,0.0027937341947108507,0.1901337057352066,0.1889117956161499,0.0035836275201290846,0.1889117956161499,,, +,,,,135,2025-12-13 13:57:32.738397,,,,,,,0.20658621191978455,0.002797299064695835,0.20658621191978455 +0.21315224468708038,0.001869451953098178,0.21315224468708038,16749.0,136,2025-12-13 14:03:22.962179,,,,,,,,, +0.2255125194787979,0.0028573295567184687,0.2255125194787979,16799.0,136,2025-12-13 14:22:21.842108,,,,,,,,, +0.17988912761211395,0.00219418085180223,0.17988912761211395,16849.0,136,2025-12-13 14:36:41.016836,,,,,,,,, +,,,,136,2025-12-13 14:44:55.395509,0.18749864399433136,0.0028984842356294394,0.18749864399433136,0.1868906319141388,0.0029556595254689455,0.1868906319141388,,, +,,,,136,2025-12-13 14:45:19.340887,,,,,,,0.20359529554843903,0.0029122312553226948,0.20359529554843903 +0.20237866044044495,0.00474149314686656,0.20237866044044495,16899.0,137,2025-12-13 14:56:27.873188,,,,,,,,, +0.21406501531600952,0.0032687727361917496,0.21406501531600952,16949.0,137,2025-12-13 15:07:49.106717,,,,,,,,, +,,,,137,2025-12-13 15:20:22.921372,0.18762867152690887,0.004233395215123892,0.18762867152690887,0.1873658299446106,0.003455297788605094,0.1873658299446106,,, +,,,,137,2025-12-13 15:20:46.750688,,,,,,,0.20490524172782898,0.0031799483112990856,0.20490524172782898 +0.16768644750118256,0.0018310246523469687,0.16768644750118256,16999.0,138,2025-12-13 15:25:40.375639,,,,,,,,, +0.21052758395671844,0.0032606946770101786,0.21052758395671844,17049.0,138,2025-12-13 15:36:09.365703,,,,,,,,, +0.17591656744480133,0.00165286916308105,0.17591656744480133,17099.0,138,2025-12-13 15:49:45.179077,,,,,,,,, +,,,,138,2025-12-13 15:58:23.926593,0.18449433147907257,0.0035209800116717815,0.18449433147907257,0.17824681103229523,0.002267596311867237,0.17824681103229523,,, +,,,,138,2025-12-13 15:58:48.123323,,,,,,,0.2040613889694214,0.0029867806006222963,0.2040613889694214 +0.18453846871852875,0.0026236672420054674,0.18453846871852875,17149.0,139,2025-12-13 16:10:59.500155,,,,,,,,, +0.19572003185749054,0.0015035574324429035,0.19572003185749054,17199.0,139,2025-12-13 16:22:11.904758,,,,,,,,, +,,,,139,2025-12-13 16:38:20.268823,0.1852438598871231,0.002365098102018237,0.1852438598871231,0.18996170163154602,0.0018069865182042122,0.18996170163154602,,, +,,,,139,2025-12-13 16:38:44.458902,,,,,,,0.20310528576374054,0.0031931111589074135,0.20310528576374054 +0.20662559568881989,0.005036249756813049,0.20662559568881989,17249.0,140,2025-12-13 16:47:24.280898,,,,,,,,, +0.20181143283843994,0.003146093338727951,0.20181143283843994,17299.0,140,2025-12-13 16:57:46.817284,,,,,,,,, +0.2009458988904953,0.001540906261652708,0.2009458988904953,17349.0,140,2025-12-13 17:07:49.841025,,,,,,,,, +,,,,140,2025-12-13 17:14:10.655198,0.18329352140426636,0.001988321077078581,0.18329352140426636,0.18990929424762726,0.0019539478234946728,0.18990929424762726,,, +,,,,140,2025-12-13 17:14:34.840413,,,,,,,0.20166169106960297,0.0032223921734839678,0.20166169106960297 +0.19431449472904205,0.0014095281949266791,0.19431449472904205,17399.0,141,2025-12-13 17:28:52.785066,,,,,,,,, +0.206987664103508,0.002962807659059763,0.206987664103508,17449.0,141,2025-12-13 17:40:07.750977,,,,,,,,, +,,,,141,2025-12-13 17:51:40.369531,0.19305208325386047,0.0032591286581009626,0.19305208325386047,0.19154886901378632,0.0035290985833853483,0.19154886901378632,,, +,,,,141,2025-12-13 17:52:04.371987,,,,,,,0.20595869421958923,0.0031064292415976524,0.20595869421958923 +0.1854657232761383,0.0013606678694486618,0.1854657232761383,17499.0,142,2025-12-13 17:59:54.685913,,,,,,,,, +0.19814859330654144,0.001591899199411273,0.19814859330654144,17549.0,142,2025-12-13 18:11:51.984707,,,,,,,,, +0.2094300091266632,0.004344117362052202,0.2094300091266632,17599.0,142,2025-12-13 18:24:50.557617,,,,,,,,, +,,,,142,2025-12-13 18:30:12.331342,0.18175597488880157,0.0019525694660842419,0.18175597488880157,0.1967010498046875,0.0033037071116268635,0.1967010498046875,,, +,,,,142,2025-12-13 18:30:37.292972,,,,,,,0.20372189581394196,0.0030920528806746006,0.20372189581394196 +0.24815556406974792,0.007014741655439138,0.24815556406974792,17649.0,143,2025-12-13 18:44:25.757322,,,,,,,,, +0.18923746049404144,0.006572770420461893,0.18923746049404144,17699.0,143,2025-12-13 18:59:51.473863,,,,,,,,, +,,,,143,2025-12-13 19:10:12.078816,0.1958920806646347,0.0023167014587670565,0.1958920806646347,0.19270330667495728,0.003569606691598892,0.19270330667495728,,, +,,,,143,2025-12-13 19:10:36.486124,,,,,,,0.20644663274288177,0.003200701903551817,0.20644663274288177 +0.16641664505004883,0.0012936026323586702,0.16641664505004883,17749.0,144,2025-12-13 19:17:10.948085,,,,,,,,, +0.24553978443145752,0.006646375637501478,0.24553978443145752,17799.0,144,2025-12-13 19:28:09.174054,,,,,,,,, +0.24046678841114044,0.005243991501629353,0.24046678841114044,17849.0,144,2025-12-13 19:38:21.966574,,,,,,,,, +,,,,144,2025-12-13 19:43:27.892929,0.18863587081432343,0.002224284689873457,0.18863587081432343,0.18989840149879456,0.001809622161090374,0.18989840149879456,,, +,,,,144,2025-12-13 19:43:53.635173,,,,,,,0.20804685354232788,0.0030308892019093037,0.20804685354232788 +0.17468112707138062,0.0022069518454372883,0.17468112707138062,17899.0,145,2025-12-13 19:58:10.656794,,,,,,,,, +0.18716682493686676,0.005307925399392843,0.18716682493686676,17949.0,145,2025-12-13 20:11:33.398292,,,,,,,,, +,,,,145,2025-12-13 20:25:19.090836,0.18423329293727875,0.002386638429015875,0.18423329293727875,0.19273334741592407,0.0033043776638805866,0.19273334741592407,,, +,,,,145,2025-12-13 20:25:44.280126,,,,,,,0.20247507095336914,0.003200127277523279,0.20247507095336914 +0.15397585928440094,0.0026369686238467693,0.15397585928440094,17999.0,146,2025-12-13 20:33:42.536572,,,,,,,,, +0.17504775524139404,0.000984499230980873,0.17504775524139404,18049.0,146,2025-12-13 20:44:35.620790,,,,,,,,, +0.1856486201286316,0.0022618293296545744,0.1856486201286316,18099.0,146,2025-12-13 20:58:43.960032,,,,,,,,, +,,,,146,2025-12-13 21:03:26.752587,0.18735772371292114,0.0024374746717512608,0.18735772371292114,0.19525378942489624,0.002823167946189642,0.19525378942489624,,, +,,,,146,2025-12-13 21:03:51.824145,,,,,,,0.2017139494419098,0.002889032242819667,0.2017139494419098 +0.1747940331697464,0.004362243693321943,0.1747940331697464,18149.0,147,2025-12-13 21:17:16.198112,,,,,,,,, +0.23199816048145294,0.0039706602692604065,0.23199816048145294,18199.0,147,2025-12-13 21:26:46.088891,,,,,,,,, +,,,,147,2025-12-13 21:43:05.191581,0.18099895119667053,0.0030040121637284756,0.18099895119667053,0.176581010222435,0.002610520925372839,0.176581010222435,,, +,,,,147,2025-12-13 21:43:54.900797,,,,,,,0.20638634264469147,0.0032360549084842205,0.20638634264469147 +0.22002041339874268,0.0051727634854614735,0.22002041339874268,18249.0,148,2025-12-13 21:51:50.023449,,,,,,,,, +0.20693136751651764,0.0015931101515889168,0.20693136751651764,18299.0,148,2025-12-13 22:05:22.425930,,,,,,,,, +0.24033689498901367,0.002812339225783944,0.24033689498901367,18349.0,148,2025-12-13 22:15:45.127685,,,,,,,,, +,,,,148,2025-12-13 22:20:07.604684,0.18810155987739563,0.003762433771044016,0.18810155987739563,0.1888524293899536,0.0036158370785415173,0.1888524293899536,,, +,,,,148,2025-12-13 22:20:31.888920,,,,,,,0.20138029754161835,0.0027584172785282135,0.20138029754161835 +0.20698139071464539,0.003922474570572376,0.20698139071464539,18399.0,149,2025-12-13 22:36:32.947079,,,,,,,,, +0.17214812338352203,0.001440767664462328,0.17214812338352203,18449.0,149,2025-12-13 22:50:35.427229,,,,,,,,, +,,,,149,2025-12-13 23:01:27.839256,0.18889585137367249,0.004568661563098431,0.18889585137367249,0.19183558225631714,0.0027858498506247997,0.19183558225631714,,, +,,,,149,2025-12-13 23:01:52.258405,,,,,,,0.2058694213628769,0.00319105782546103,0.2058694213628769 +0.25591933727264404,0.005133237224072218,0.25591933727264404,18499.0,150,2025-12-13 23:09:38.078197,,,,,,,,, +0.21256202459335327,0.0028850098606199026,0.21256202459335327,18549.0,150,2025-12-13 23:26:23.635506,,,,,,,,, +0.2016627937555313,0.00158958591055125,0.2016627937555313,18599.0,150,2025-12-13 23:38:22.145144,,,,,,,,, +,,,,150,2025-12-13 23:42:33.677991,0.18201979994773865,0.0028996262699365616,0.18201979994773865,0.19249887764453888,0.0034550216514617205,0.19249887764453888,,, +,,,,150,2025-12-13 23:42:58.098122,,,,,,,0.20456497371196747,0.0031900980975478888,0.20456497371196747 +0.22443431615829468,0.0015738628571853042,0.22443431615829468,18649.0,151,2025-12-13 23:59:11.037275,,,,,,,,, +0.21125616133213043,0.0023162479046732187,0.21125616133213043,18699.0,151,2025-12-14 00:13:37.838077,,,,,,,,, +,,,,151,2025-12-14 00:23:03.786812,0.17528381943702698,0.0022239701356738806,0.17528381943702698,0.1927787959575653,0.0025148396380245686,0.1927787959575653,,, +,,,,151,2025-12-14 00:23:27.672790,,,,,,,0.20748265087604523,0.002959341276437044,0.20748265087604523 +0.27607855200767517,0.005154413171112537,0.27607855200767517,18749.0,152,2025-12-14 00:33:48.008253,,,,,,,,, +0.1959124207496643,0.0050796400755643845,0.1959124207496643,18799.0,152,2025-12-14 00:49:01.052204,,,,,,,,, +,,,,152,2025-12-14 01:09:55.087959,0.19221927225589752,0.0030216549057513475,0.19221927225589752,0.192337766289711,0.002060649683699012,0.192337766289711,,, +,,,,152,2025-12-14 01:10:19.510173,,,,,,,0.2089400738477707,0.0033798436634242535,0.2089400738477707 +0.19660437107086182,0.001962269889190793,0.19660437107086182,18849.0,153,2025-12-14 01:11:57.787356,,,,,,,,, +0.20249532163143158,0.0024601216427981853,0.20249532163143158,18899.0,153,2025-12-14 01:26:54.215132,,,,,,,,, +0.22463400661945343,0.001359146204777062,0.22463400661945343,18949.0,153,2025-12-14 01:41:09.750408,,,,,,,,, +,,,,153,2025-12-14 01:51:01.806243,0.18848200142383575,0.00276883109472692,0.18848200142383575,0.19076812267303467,0.003036884590983391,0.19076812267303467,,, +,,,,153,2025-12-14 01:51:25.861480,,,,,,,0.20744828879833221,0.002928397385403514,0.20744828879833221 +0.15656347572803497,0.0008911942131817341,0.15656347572803497,18999.0,154,2025-12-14 02:00:59.747469,,,,,,,,, +0.1955457180738449,0.001889366889372468,0.1955457180738449,19049.0,154,2025-12-14 02:16:13.412120,,,,,,,,, +,,,,154,2025-12-14 02:31:44.843900,0.1931775063276291,0.003233040450140834,0.1931775063276291,0.19011925160884857,0.0019113338785246015,0.19011925160884857,,, +,,,,154,2025-12-14 02:32:09.449416,,,,,,,0.2031787633895874,0.0030174979474395514,0.2031787633895874 +0.24254591763019562,0.003594502341002226,0.24254591763019562,19099.0,155,2025-12-14 02:34:52.163621,,,,,,,,, +0.20740097761154175,0.0032068786676973104,0.20740097761154175,19149.0,155,2025-12-14 02:49:13.189382,,,,,,,,, +0.22820892930030823,0.005643394310027361,0.22820892930030823,19199.0,155,2025-12-14 03:03:03.691457,,,,,,,,, +,,,,155,2025-12-14 03:13:07.426389,0.1919104903936386,0.003349195932969451,0.1919104903936386,0.18699811398983002,0.002044464461505413,0.18699811398983002,,, +,,,,155,2025-12-14 03:13:32.294103,,,,,,,0.2034289836883545,0.002985813422128558,0.2034289836883545 +0.1674298197031021,0.004585047718137503,0.1674298197031021,19249.0,156,2025-12-14 03:26:09.700182,,,,,,,,, +0.1939510703086853,0.003056705230847001,0.1939510703086853,19299.0,156,2025-12-14 03:38:30.423020,,,,,,,,, +,,,,156,2025-12-14 03:52:05.116893,0.18588408827781677,0.002690088702365756,0.18588408827781677,0.19404804706573486,0.002495205495506525,0.19404804706573486,,, +,,,,156,2025-12-14 03:52:29.533107,,,,,,,0.20112788677215576,0.0029418824706226587,0.20112788677215576 +0.2058226615190506,0.0020007400307804346,0.2058226615190506,19349.0,157,2025-12-14 03:57:19.412175,,,,,,,,, +0.2346084713935852,0.005666627548635006,0.2346084713935852,19399.0,157,2025-12-14 04:10:51.981896,,,,,,,,, +0.23106428980827332,0.006460053846240044,0.23106428980827332,19449.0,157,2025-12-14 04:20:45.023292,,,,,,,,, +,,,,157,2025-12-14 04:28:28.662541,0.19217100739479065,0.002502773655578494,0.19217100739479065,0.19014418125152588,0.0028309153858572245,0.19014418125152588,,, +,,,,157,2025-12-14 04:28:53.185979,,,,,,,0.2055845558643341,0.003016236238181591,0.2055845558643341 +0.1917736977338791,0.0016348008066415787,0.1917736977338791,19499.0,158,2025-12-14 04:44:29.207416,,,,,,,,, +0.17747405171394348,0.002150770975276828,0.17747405171394348,19549.0,158,2025-12-14 04:54:08.428344,,,,,,,,, +,,,,158,2025-12-14 05:07:22.509884,0.1877669245004654,0.0029114042408764362,0.1877669245004654,0.19466085731983185,0.0033860330004245043,0.19466085731983185,,, +,,,,158,2025-12-14 05:07:46.715988,,,,,,,0.2065860480070114,0.0029998719692230225,0.2065860480070114 +0.22980889678001404,0.0022846022620797157,0.22980889678001404,19599.0,159,2025-12-14 05:12:52.437546,,,,,,,,, +0.18680009245872498,0.0013080628123134375,0.18680009245872498,19649.0,159,2025-12-14 05:25:32.698528,,,,,,,,, +0.16021814942359924,0.0013227197341620922,0.16021814942359924,19699.0,159,2025-12-14 05:44:17.086137,,,,,,,,, +,,,,159,2025-12-14 05:51:04.452096,0.18987485766410828,0.002912286901846528,0.18987485766410828,0.18597953021526337,0.0022794189862906933,0.18597953021526337,,, +,,,,159,2025-12-14 05:51:28.122611,,,,,,,0.19948942959308624,0.0025980621576309204,0.19948942959308624 +0.18413245677947998,0.002141786040738225,0.18413245677947998,19749.0,160,2025-12-14 06:03:07.473253,,,,,,,,, +0.17797982692718506,0.0028098493348807096,0.17797982692718506,19799.0,160,2025-12-14 06:12:25.368488,,,,,,,,, +,,,,160,2025-12-14 06:27:35.030540,0.18497171998023987,0.0036815498024225235,0.18497171998023987,0.19119694828987122,0.0043425424955785275,0.19119694828987122,,, +,,,,160,2025-12-14 06:27:59.466142,,,,,,,0.20301885902881622,0.0030863445717841387,0.20301885902881622 +0.16249129176139832,0.0009987694211304188,0.16249129176139832,19849.0,161,2025-12-14 06:32:35.520865,,,,,,,,, +0.21673259139060974,0.002501680050045252,0.21673259139060974,19899.0,161,2025-12-14 06:44:23.471636,,,,,,,,, +0.19635173678398132,0.004956824705004692,0.19635173678398132,19949.0,161,2025-12-14 06:57:07.704317,,,,,,,,, +,,,,161,2025-12-14 07:03:34.679258,0.1854834258556366,0.003520736936479807,0.1854834258556366,0.1906592696905136,0.002705790800973773,0.1906592696905136,,, +,,,,161,2025-12-14 07:03:58.466463,,,,,,,0.20558509230613708,0.003520337166264653,0.20558509230613708 +0.21485258638858795,0.003676459426060319,0.21485258638858795,19999.0,162,2025-12-14 07:16:40.816069,,,,,,,,, +0.22243477404117584,0.0020200619474053383,0.22243477404117584,20049.0,162,2025-12-14 07:30:54.683988,,,,,,,,, +,,,,162,2025-12-14 07:42:40.598509,0.19111202657222748,0.002812022343277931,0.19111202657222748,0.18293088674545288,0.003029734129086137,0.18293088674545288,,, +,,,,162,2025-12-14 07:43:05.152385,,,,,,,0.2008288949728012,0.00292504602111876,0.2008288949728012 +0.20642130076885223,0.00211900332942605,0.20642130076885223,20099.0,163,2025-12-14 07:47:40.018568,,,,,,,,, +0.21226713061332703,0.008885057643055916,0.21226713061332703,20149.0,163,2025-12-14 07:58:38.607724,,,,,,,,, +0.23535893857479095,0.003550810506567359,0.23535893857479095,20199.0,163,2025-12-14 08:11:58.129403,,,,,,,,, +,,,,163,2025-12-14 08:20:21.294437,0.18554802238941193,0.0026901811361312866,0.18554802238941193,0.1894339919090271,0.003331280779093504,0.1894339919090271,,, +,,,,163,2025-12-14 08:20:45.554108,,,,,,,0.2004864513874054,0.003070739796385169,0.2004864513874054 +0.1815943717956543,0.003973413724452257,0.1815943717956543,20249.0,164,2025-12-14 08:34:27.500332,,,,,,,,, +0.1708163470029831,0.0026830467395484447,0.1708163470029831,20299.0,164,2025-12-14 08:45:53.632157,,,,,,,,, +,,,,164,2025-12-14 08:59:00.895529,0.1809915006160736,0.002497266046702862,0.1809915006160736,0.19515420496463776,0.0034187694545835257,0.19515420496463776,,, +,,,,164,2025-12-14 08:59:25.670356,,,,,,,0.20180895924568176,0.0028170556761324406,0.20180895924568176 +0.17540137469768524,0.0011756778694689274,0.17540137469768524,20349.0,165,2025-12-14 09:08:27.359274,,,,,,,,, +0.17217977344989777,0.0014963365392759442,0.17217977344989777,20399.0,165,2025-12-14 09:18:39.251208,,,,,,,,, +0.20117169618606567,0.0020489851012825966,0.20117169618606567,20449.0,165,2025-12-14 09:31:36.837530,,,,,,,,, +,,,,165,2025-12-14 09:38:09.236606,0.186709463596344,0.003479348961263895,0.186709463596344,0.19117414951324463,0.002796400338411331,0.19117414951324463,,, +,,,,165,2025-12-14 09:38:34.194995,,,,,,,0.2053021490573883,0.003160986816510558,0.2053021490573883 +0.20310749113559723,0.00162974139675498,0.20310749113559723,20499.0,166,2025-12-14 09:52:01.274135,,,,,,,,, +0.16286198794841766,0.001361771603114903,0.16286198794841766,20549.0,166,2025-12-14 10:02:36.387104,,,,,,,,, +,,,,166,2025-12-14 10:12:26.840360,0.18826884031295776,0.0020237683784216642,0.18826884031295776,0.186753049492836,0.0019431994296610355,0.186753049492836,,, +,,,,166,2025-12-14 10:12:49.741077,,,,,,,0.20549586415290833,0.003166372422128916,0.20549586415290833 +0.21119976043701172,0.0015190516132861376,0.21119976043701172,20599.0,167,2025-12-14 10:20:10.399051,,,,,,,,, +0.2159600406885147,0.006179798394441605,0.2159600406885147,20649.0,167,2025-12-14 10:31:16.260414,,,,,,,,, +0.22254472970962524,0.005467198323458433,0.22254472970962524,20699.0,167,2025-12-14 10:40:28.076688,,,,,,,,, +,,,,167,2025-12-14 10:45:49.072777,0.19317585229873657,0.0031945263035595417,0.19317585229873657,0.18614442646503448,0.002168570877984166,0.18614442646503448,,, +,,,,167,2025-12-14 10:46:13.751449,,,,,,,0.20197534561157227,0.0030640761833637953,0.20197534561157227 +0.18255290389060974,0.0020740521140396595,0.18255290389060974,20749.0,168,2025-12-14 11:00:40.704044,,,,,,,,, +0.23289687931537628,0.0024901083670556545,0.23289687931537628,20799.0,168,2025-12-14 11:13:43.718814,,,,,,,,, +,,,,168,2025-12-14 11:24:44.194250,0.1877778023481369,0.002964314306154847,0.1877778023481369,0.18330565094947815,0.0032374276779592037,0.18330565094947815,,, +,,,,168,2025-12-14 11:25:08.885611,,,,,,,0.20285773277282715,0.0031447543296962976,0.20285773277282715 +0.26743972301483154,0.010689916089177132,0.26743972301483154,20849.0,169,2025-12-14 11:33:31.368488,,,,,,,,, +0.20287993550300598,0.0034757424145936966,0.20287993550300598,20899.0,169,2025-12-14 11:48:50.731876,,,,,,,,, +0.21189409494400024,0.0022505351807922125,0.21189409494400024,20949.0,169,2025-12-14 11:58:40.035675,,,,,,,,, +,,,,169,2025-12-14 12:03:41.044461,0.18827137351036072,0.002157816430553794,0.18827137351036072,0.18390195071697235,0.0028543444350361824,0.18390195071697235,,, +,,,,169,2025-12-14 12:04:05.356431,,,,,,,0.20184552669525146,0.0027691770810633898,0.20184552669525146 +0.18459786474704742,0.003781083272770047,0.18459786474704742,20999.0,170,2025-12-14 12:18:35.723297,,,,,,,,, +0.18384872376918793,0.0013122003292664886,0.18384872376918793,21049.0,170,2025-12-14 12:30:56.223556,,,,,,,,, +,,,,170,2025-12-14 12:42:38.367910,0.18692179024219513,0.002305918838828802,0.18692179024219513,0.18490873277187347,0.003783494932577014,0.18490873277187347,,, +,,,,170,2025-12-14 12:43:03.204477,,,,,,,0.19979408383369446,0.0030132352840155363,0.19979408383369446 +0.21190747618675232,0.005497168283909559,0.21190747618675232,21099.0,171,2025-12-14 12:50:10.807782,,,,,,,,, +0.21196924149990082,0.006779488176107407,0.21196924149990082,21149.0,171,2025-12-14 13:01:35.543433,,,,,,,,, +0.1815924346446991,0.001584401703439653,0.1815924346446991,21199.0,171,2025-12-14 13:14:10.537863,,,,,,,,, +,,,,171,2025-12-14 13:18:55.398198,0.17858050763607025,0.0029920097440481186,0.17858050763607025,0.18223333358764648,0.002069239504635334,0.18223333358764648,,, +,,,,171,2025-12-14 13:19:19.906405,,,,,,,0.20403721928596497,0.003196412930265069,0.20403721928596497 +0.16475555300712585,0.001015497138723731,0.16475555300712585,21249.0,172,2025-12-14 13:31:47.570923,,,,,,,,, +0.22982320189476013,0.005029649008065462,0.22982320189476013,21299.0,172,2025-12-14 13:41:53.809236,,,,,,,,, +,,,,172,2025-12-14 13:58:23.602681,0.17478804290294647,0.003304622368887067,0.17478804290294647,0.1803770512342453,0.0020794912707060575,0.1803770512342453,,, +,,,,172,2025-12-14 13:58:48.029317,,,,,,,0.20370401442050934,0.003196980804204941,0.20370401442050934 +0.19780442118644714,0.008045661263167858,0.19780442118644714,21349.0,173,2025-12-14 14:07:27.383465,,,,,,,,, +0.23103506863117218,0.0033180753234773874,0.23103506863117218,21399.0,173,2025-12-14 14:16:31.990469,,,,,,,,, +0.21468275785446167,0.001844961429014802,0.21468275785446167,21449.0,173,2025-12-14 14:31:01.162780,,,,,,,,, +,,,,173,2025-12-14 14:35:25.156762,0.1878022700548172,0.0048406170681118965,0.1878022700548172,0.1951896697282791,0.003992184065282345,0.1951896697282791,,, +,,,,173,2025-12-14 14:35:50.543891,,,,,,,0.20198526978492737,0.0027778693474829197,0.20198526978492737 +0.1580265909433365,0.0029930570162832737,0.1580265909433365,21499.0,174,2025-12-14 14:54:22.825162,,,,,,,,, +0.16621164977550507,0.0008624477777630091,0.16621164977550507,21549.0,174,2025-12-14 15:07:08.720393,,,,,,,,, +,,,,174,2025-12-14 15:20:30.815172,0.18368931114673615,0.002516777254641056,0.18368931114673615,0.18826715648174286,0.002949285786598921,0.18826715648174286,,, +,,,,174,2025-12-14 15:20:56.403350,,,,,,,0.20184963941574097,0.00305035593919456,0.20184963941574097 +0.19451217353343964,0.006801022216677666,0.19451217353343964,21599.0,175,2025-12-14 15:30:45.733085,,,,,,,,, +0.22722581028938293,0.0014651321107521653,0.22722581028938293,21649.0,175,2025-12-14 15:42:31.099807,,,,,,,,, +0.2189999222755432,0.005164593458175659,0.2189999222755432,21699.0,175,2025-12-14 15:51:36.090235,,,,,,,,, +,,,,175,2025-12-14 15:55:40.212214,0.17902210354804993,0.0026476953644305468,0.17902210354804993,0.17228323221206665,0.0023276889696717262,0.17228323221206665,,, +,,,,175,2025-12-14 15:56:25.684150,,,,,,,0.2048397958278656,0.003498725825920701,0.2048397958278656 +0.19427058100700378,0.002083885483443737,0.19427058100700378,21749.0,176,2025-12-14 16:14:19.554094,,,,,,,,, +0.18555034697055817,0.0016958214109763503,0.18555034697055817,21799.0,176,2025-12-14 16:26:24.864231,,,,,,,,, +,,,,176,2025-12-14 16:34:39.966436,0.18198341131210327,0.0030313332099467516,0.18198341131210327,0.18069268763065338,0.0025199425872415304,0.18069268763065338,,, +,,,,176,2025-12-14 16:35:05.864998,,,,,,,0.20025984942913055,0.002853446640074253,0.20025984942913055 +0.20096106827259064,0.005079533904790878,0.20096106827259064,21849.0,177,2025-12-14 16:45:54.613442,,,,,,,,, +0.202921524643898,0.0013997826026752591,0.202921524643898,21899.0,177,2025-12-14 16:57:57.115169,,,,,,,,, +,,,,177,2025-12-14 17:15:05.357470,0.1852678656578064,0.0027919940184801817,0.1852678656578064,0.18751969933509827,0.00302363489754498,0.18751969933509827,,, +,,,,177,2025-12-14 17:15:28.838568,,,,,,,0.2017042487859726,0.0029313876293599606,0.2017042487859726 +0.2080961912870407,0.004954738076776266,0.2080961912870407,21949.0,178,2025-12-14 17:19:36.363813,,,,,,,,, +0.25436851382255554,0.0031478300224989653,0.25436851382255554,21999.0,178,2025-12-14 17:30:18.531785,,,,,,,,, +0.19909848272800446,0.0021137006115168333,0.19909848272800446,22049.0,178,2025-12-14 17:40:51.168189,,,,,,,,, +,,,,178,2025-12-14 17:50:59.422093,0.182587668299675,0.002051640534773469,0.182587668299675,0.17797943949699402,0.0022322037257254124,0.17797943949699402,,, +,,,,178,2025-12-14 17:51:23.649116,,,,,,,0.20113252103328705,0.003041694173589349,0.20113252103328705 +0.23488210141658783,0.0038517192006111145,0.23488210141658783,22099.0,179,2025-12-14 18:00:36.811248,,,,,,,,, +0.23313197493553162,0.006938680075109005,0.23313197493553162,22149.0,179,2025-12-14 18:12:09.112072,,,,,,,,, +,,,,179,2025-12-14 18:29:40.383787,0.18476776778697968,0.003904481651261449,0.18476776778697968,0.18858879804611206,0.0024840065743774176,0.18858879804611206,,, +,,,,179,2025-12-14 18:30:04.290492,,,,,,,0.19783420860767365,0.0028780996799468994,0.19783420860767365 +0.2223811149597168,0.003875160124152899,0.2223811149597168,22199.0,180,2025-12-14 18:32:48.448432,,,,,,,,, +0.18106713891029358,0.00107187672983855,0.18106713891029358,22249.0,180,2025-12-14 18:48:52.815978,,,,,,,,, +0.1913534551858902,0.0025038928724825382,0.1913534551858902,22299.0,180,2025-12-14 19:00:05.094098,,,,,,,,, +,,,,180,2025-12-14 19:09:42.150827,0.17711202800273895,0.0021976218558847904,0.17711202800273895,0.19301319122314453,0.004216330125927925,0.19301319122314453,,, +,,,,180,2025-12-14 19:10:06.338708,,,,,,,0.2022445946931839,0.0029521279502660036,0.2022445946931839 +0.21022668480873108,0.0041609955951571465,0.21022668480873108,22349.0,181,2025-12-14 19:18:23.669318,,,,,,,,, +0.22526216506958008,0.0020235071424394846,0.22526216506958008,22399.0,181,2025-12-14 19:29:48.064609,,,,,,,,, +,,,,181,2025-12-14 19:46:36.881791,0.1808009147644043,0.0024632865097373724,0.1808009147644043,0.19077084958553314,0.0022513552103191614,0.19077084958553314,,, +,,,,181,2025-12-14 19:47:01.293700,,,,,,,0.20320181548595428,0.003096562111750245,0.20320181548595428 +0.1957409530878067,0.0015502269379794598,0.1957409530878067,22449.0,182,2025-12-14 19:51:09.248009,,,,,,,,, +0.20105507969856262,0.006540749687701464,0.20105507969856262,22499.0,182,2025-12-14 20:05:54.164563,,,,,,,,, +0.21976742148399353,0.008606594987213612,0.21976742148399353,22549.0,182,2025-12-14 20:22:18.638679,,,,,,,,, +,,,,182,2025-12-14 20:29:27.650737,0.17717571556568146,0.0021280325017869473,0.17717571556568146,0.1869022250175476,0.0030404573772102594,0.1869022250175476,,, +,,,,182,2025-12-14 20:29:54.761189,,,,,,,0.20112337172031403,0.0029284392949193716,0.20112337172031403 +0.23593930900096893,0.002599257742986083,0.23593930900096893,22599.0,183,2025-12-14 20:41:48.819600,,,,,,,,, +0.2070479691028595,0.0021078085992485285,0.2070479691028595,22649.0,183,2025-12-14 20:53:20.113203,,,,,,,,, +,,,,183,2025-12-14 21:08:49.556121,0.18468166887760162,0.002357523189857602,0.18468166887760162,0.1859128177165985,0.0024981065653264523,0.1859128177165985,,, +,,,,183,2025-12-14 21:09:14.029379,,,,,,,0.19843927025794983,0.0029665466863662004,0.19843927025794983 +0.18617746233940125,0.0034268538001924753,0.18617746233940125,22699.0,184,2025-12-14 21:12:26.337323,,,,,,,,, +0.1583113670349121,0.0045892843045294285,0.1583113670349121,22749.0,184,2025-12-14 21:23:47.527133,,,,,,,,, +0.21148671209812164,0.0013384876074269414,0.21148671209812164,22799.0,184,2025-12-14 21:35:36.296442,,,,,,,,, +,,,,184,2025-12-14 21:42:20.459640,0.17521101236343384,0.002473871922120452,0.17521101236343384,0.1894984245300293,0.0026221233420073986,0.1894984245300293,,, +,,,,184,2025-12-14 21:42:45.288357,,,,,,,0.198618084192276,0.003052067942917347,0.198618084192276 +0.1988760083913803,0.0015348979504778981,0.1988760083913803,22849.0,185,2025-12-14 21:53:21.585163,,,,,,,,, +0.15869344770908356,0.001922018826007843,0.15869344770908356,22899.0,185,2025-12-14 22:02:34.150755,,,,,,,,, +,,,,185,2025-12-14 22:19:53.294753,0.1846526861190796,0.002312467200681567,0.1846526861190796,0.1883329302072525,0.003267803229391575,0.1883329302072525,,, +,,,,185,2025-12-14 22:20:18.572448,,,,,,,0.1998288482427597,0.002654512645676732,0.1998288482427597 +0.22310870885849,0.00434449315071106,0.22310870885849,22949.0,186,2025-12-14 22:23:33.846029,,,,,,,,, +0.23432089388370514,0.007402549032121897,0.23432089388370514,22999.0,186,2025-12-14 22:35:31.508319,,,,,,,,, +0.17500056326389313,0.004819471389055252,0.17500056326389313,23049.0,186,2025-12-14 22:55:15.268142,,,,,,,,, +,,,,186,2025-12-14 23:01:35.076157,0.179201140999794,0.0032262748572975397,0.179201140999794,0.1821247786283493,0.0024080772418528795,0.1821247786283493,,, +,,,,186,2025-12-14 23:01:59.143867,,,,,,,0.20228029787540436,0.003233585972338915,0.20228029787540436 +0.23599010705947876,0.005490089301019907,0.23599010705947876,23099.0,187,2025-12-14 23:16:15.340826,,,,,,,,, +0.18614661693572998,0.003557870164513588,0.18614661693572998,23149.0,187,2025-12-14 23:28:44.099644,,,,,,,,, +,,,,187,2025-12-14 23:41:32.804038,0.1891160011291504,0.0028624683618545532,0.1891160011291504,0.1903827041387558,0.003984532784670591,0.1903827041387558,,, +,,,,187,2025-12-14 23:41:57.186225,,,,,,,0.2015029489994049,0.0029768389649689198,0.2015029489994049 +0.20619720220565796,0.002256285399198532,0.20619720220565796,23199.0,188,2025-12-14 23:47:21.576054,,,,,,,,, +0.173837348818779,0.0010623288108035922,0.173837348818779,23249.0,188,2025-12-14 23:59:53.133443,,,,,,,,, +0.1842641979455948,0.004681156016886234,0.1842641979455948,23299.0,188,2025-12-15 00:13:34.347717,,,,,,,,, +,,,,188,2025-12-15 00:19:43.483732,0.19443194568157196,0.00245118560269475,0.19443194568157196,0.18404467403888702,0.002003428526222706,0.18404467403888702,,, +,,,,188,2025-12-15 00:20:08.308790,,,,,,,0.20004330575466156,0.002926879795268178,0.20004330575466156 +0.22452834248542786,0.0017659361474215984,0.22452834248542786,23349.0,189,2025-12-15 00:31:56.584809,,,,,,,,, +0.1786302626132965,0.0041792490519583225,0.1786302626132965,23399.0,189,2025-12-15 00:44:27.397848,,,,,,,,, +,,,,189,2025-12-15 00:58:21.201269,0.18018241226673126,0.00209776870906353,0.18018241226673126,0.19179794192314148,0.0025181490927934647,0.19179794192314148,,, +,,,,189,2025-12-15 00:58:45.344541,,,,,,,0.20044267177581787,0.0032054646871984005,0.20044267177581787 +0.1885627806186676,0.0012409182963892817,0.1885627806186676,23449.0,190,2025-12-15 01:05:07.265198,,,,,,,,, +0.19101311266422272,0.001247909152880311,0.19101311266422272,23499.0,190,2025-12-15 01:15:36.953985,,,,,,,,, +0.17359060049057007,0.005128961056470871,0.17359060049057007,23549.0,190,2025-12-15 01:26:38.495173,,,,,,,,, +,,,,190,2025-12-15 01:32:33.772601,0.1891593337059021,0.0031432555988430977,0.1891593337059021,0.18823643028736115,0.002500977134332061,0.18823643028736115,,, +,,,,190,2025-12-15 01:32:57.945374,,,,,,,0.20014524459838867,0.0033304921817034483,0.20014524459838867 +0.19241784512996674,0.001497366582043469,0.19241784512996674,23599.0,191,2025-12-15 01:47:01.552609,,,,,,,,, +0.2056940346956253,0.0016335430555045605,0.2056940346956253,23649.0,191,2025-12-15 01:57:45.164139,,,,,,,,, +,,,,191,2025-12-15 02:08:34.687409,0.18162131309509277,0.002042928012087941,0.18162131309509277,0.1801203489303589,0.0025009005330502987,0.1801203489303589,,, +,,,,191,2025-12-15 02:08:59.118977,,,,,,,0.19721411168575287,0.003130632918328047,0.19721411168575287 +0.20379243791103363,0.00516984798014164,0.20379243791103363,23699.0,192,2025-12-15 02:13:55.194444,,,,,,,,, +0.2246471345424652,0.002393510425463319,0.2246471345424652,23749.0,192,2025-12-15 02:25:09.880298,,,,,,,,, +0.15611763298511505,0.0018595663132146,0.15611763298511505,23799.0,192,2025-12-15 02:34:49.716063,,,,,,,,, +,,,,192,2025-12-15 02:41:57.871022,0.19726377725601196,0.003138179425150156,0.19726377725601196,0.18810878694057465,0.0036564941983669996,0.18810878694057465,,, +,,,,192,2025-12-15 02:42:22.814486,,,,,,,0.20016537606716156,0.002824274590238929,0.20016537606716156 +0.18052543699741364,0.002970231929793954,0.18052543699741364,23849.0,193,2025-12-15 02:56:00.737222,,,,,,,,, +0.1767200529575348,0.001444395398721099,0.1767200529575348,23899.0,193,2025-12-15 03:06:15.670346,,,,,,,,, +,,,,193,2025-12-15 03:18:39.060793,0.17477811872959137,0.0023326477967202663,0.17477811872959137,0.18681105971336365,0.002791079692542553,0.18681105971336365,,, +,,,,193,2025-12-15 03:19:04.654243,,,,,,,0.20103009045124054,0.0029625818133354187,0.20103009045124054 +0.16950133442878723,0.0011144580785185099,0.16950133442878723,23949.0,194,2025-12-15 03:26:00.936357,,,,,,,,, +0.18110479414463043,0.0026757747400552034,0.18110479414463043,23999.0,194,2025-12-15 03:37:30.278255,,,,,,,,, +0.23664474487304688,0.005180140025913715,0.23664474487304688,24049.0,194,2025-12-15 03:47:01.512276,,,,,,,,, +,,,,194,2025-12-15 03:53:03.450922,0.1871265470981598,0.0023602256551384926,0.1871265470981598,0.17900261282920837,0.0031958427280187607,0.17900261282920837,,, +,,,,194,2025-12-15 03:53:28.221233,,,,,,,0.1963524967432022,0.0028151636943221092,0.1963524967432022 +0.2321924865245819,0.0025659212842583656,0.2321924865245819,24099.0,195,2025-12-15 04:11:26.863851,,,,,,,,, +0.2164270132780075,0.005437880754470825,0.2164270132780075,24149.0,195,2025-12-15 04:23:11.478515,,,,,,,,, +,,,,195,2025-12-15 04:33:28.433891,0.18858665227890015,0.0034539748448878527,0.18858665227890015,0.18982069194316864,0.002111711772158742,0.18982069194316864,,, +,,,,195,2025-12-15 04:33:53.401194,,,,,,,0.20400093495845795,0.003324981080368161,0.20400093495845795 +0.17046014964580536,0.0016558229690417647,0.17046014964580536,24199.0,196,2025-12-15 04:44:58.830311,,,,,,,,, +0.19583921134471893,0.0014924429124221206,0.19583921134471893,24249.0,196,2025-12-15 04:55:22.239012,,,,,,,,, +0.17186298966407776,0.001120032393373549,0.17186298966407776,24299.0,196,2025-12-15 05:05:59.493107,,,,,,,,, +,,,,196,2025-12-15 05:11:41.648400,0.18308791518211365,0.003421179950237274,0.18308791518211365,0.18369773030281067,0.002175038680434227,0.18369773030281067,,, +,,,,196,2025-12-15 05:12:05.078057,,,,,,,0.20130492746829987,0.0032723720651119947,0.20130492746829987 +0.1721705049276352,0.0017286705551669002,0.1721705049276352,24349.0,197,2025-12-15 05:29:17.902428,,,,,,,,, +0.19042013585567474,0.005620559677481651,0.19042013585567474,24399.0,197,2025-12-15 05:40:05.054849,,,,,,,,, +,,,,197,2025-12-15 05:50:54.620359,0.18244439363479614,0.002193444175645709,0.18244439363479614,0.18230211734771729,0.003504624590277672,0.18230211734771729,,, +,,,,197,2025-12-15 05:51:19.489757,,,,,,,0.20060397684574127,0.003417207859456539,0.20060397684574127 +0.19292958080768585,0.003969596698880196,0.19292958080768585,24449.0,198,2025-12-15 05:58:41.835844,,,,,,,,, +0.1758202463388443,0.0011494120117276907,0.1758202463388443,24499.0,198,2025-12-15 06:12:42.796051,,,,,,,,, +0.21437965333461761,0.006346496753394604,0.21437965333461761,24549.0,198,2025-12-15 06:23:18.564619,,,,,,,,, +,,,,198,2025-12-15 06:27:55.434505,0.18371985852718353,0.0035836203023791313,0.18371985852718353,0.18136171996593475,0.0033382417168468237,0.18136171996593475,,, +,,,,198,2025-12-15 06:28:21.642715,,,,,,,0.2043267786502838,0.0033121597953140736,0.2043267786502838 +0.1995043158531189,0.001794723211787641,0.1995043158531189,24599.0,199,2025-12-15 06:45:12.419807,,,,,,,,, +0.18615899980068207,0.0028920925687998533,0.18615899980068207,24649.0,199,2025-12-15 06:58:45.121804,,,,,,,,, +,,,,199,2025-12-15 07:11:03.532870,0.18906275928020477,0.0032932402100414038,0.18906275928020477,0.1839364916086197,0.0023956801742315292,0.1839364916086197,,, +,,,,199,2025-12-15 07:11:27.130668,,,,,,,0.2014499306678772,0.0031066846568137407,0.2014499306678772 +0.21058893203735352,0.0036199362948536873,0.21058893203735352,24699.0,200,2025-12-15 07:21:01.181826,,,,,,,,, +0.2025299072265625,0.0012167207896709442,0.2025299072265625,24749.0,200,2025-12-15 07:34:35.587660,,,,,,,,, +0.22140394151210785,0.001587153528816998,0.22140394151210785,24799.0,200,2025-12-15 07:48:47.336138,,,,,,,,, +,,,,200,2025-12-15 07:52:49.891686,0.18087680637836456,0.0019003982888534665,0.18087680637836456,0.18299968540668488,0.002451772103086114,0.18299968540668488,,, +,,,,200,2025-12-15 07:53:15.266035,,,,,,,0.2011924833059311,0.003009341424331069,0.2011924833059311 +0.19985222816467285,0.0012560022296383977,0.19985222816467285,24849.0,201,2025-12-15 08:08:17.660309,,,,,,,,, +0.19294936954975128,0.0019483693176880479,0.19294936954975128,24899.0,201,2025-12-15 08:21:02.996886,,,,,,,,, +,,,,201,2025-12-15 08:30:07.017885,0.1932334005832672,0.0035274953115731478,0.1932334005832672,0.1902780532836914,0.00393798528239131,0.1902780532836914,,, +,,,,201,2025-12-15 08:30:31.766638,,,,,,,0.20118074119091034,0.0032113040797412395,0.20118074119091034 +0.21988505125045776,0.0022266267333179712,0.21988505125045776,24949.0,202,2025-12-15 08:38:48.478640,,,,,,,,, +0.1975848525762558,0.0017745019868016243,0.1975848525762558,24999.0,202,2025-12-15 08:52:56.705912,,,,,,,,, +,,,,202,2025-12-15 09:05:30.130346,0.17851649224758148,0.002408606233075261,0.17851649224758148,0.1876365840435028,0.0021705771796405315,0.1876365840435028,,, +,,,,202,2025-12-15 09:05:55.777431,,,,,,,0.19411754608154297,0.003086488228291273,0.19411754608154297 +0.21709585189819336,0.0025743842124938965,0.21709585189819336,25049.0,203,2025-12-15 09:09:05.331888,,,,,,,,, +0.22237052023410797,0.005444251000881195,0.22237052023410797,25099.0,203,2025-12-15 09:24:53.530610,,,,,,,,, +0.1836266666650772,0.0059937164187431335,0.1836266666650772,25149.0,203,2025-12-15 09:35:47.982804,,,,,,,,, +,,,,203,2025-12-15 09:46:13.385305,0.1834205985069275,0.0033353730104863644,0.1834205985069275,0.174321249127388,0.0024354232009500265,0.174321249127388,,, +,,,,203,2025-12-15 09:47:00.738398,,,,,,,0.20118175446987152,0.0033167412038892508,0.20118175446987152 +0.17928564548492432,0.005916245747357607,0.17928564548492432,25199.0,204,2025-12-15 09:59:14.702071,,,,,,,,, +0.23450635373592377,0.005387827288359404,0.23450635373592377,25249.0,204,2025-12-15 10:10:58.692248,,,,,,,,, +,,,,204,2025-12-15 10:27:00.886448,0.18215326964855194,0.002666905988007784,0.18215326964855194,0.18473239243030548,0.0032656905241310596,0.18473239243030548,,, +,,,,204,2025-12-15 10:27:26.034690,,,,,,,0.2042621225118637,0.00312495487742126,0.2042621225118637 +0.20183052122592926,0.004758557304739952,0.20183052122592926,25299.0,205,2025-12-15 10:30:08.034890,,,,,,,,, +0.25769299268722534,0.006048588082194328,0.25769299268722534,25349.0,205,2025-12-15 10:47:20.774159,,,,,,,,, +0.21705879271030426,0.001286155078560114,0.21705879271030426,25399.0,205,2025-12-15 10:59:54.284566,,,,,,,,, +,,,,205,2025-12-15 11:10:03.363330,0.18388375639915466,0.003687986172735691,0.18388375639915466,0.1925096958875656,0.0034192795865237713,0.1925096958875656,,, +,,,,205,2025-12-15 11:10:28.278417,,,,,,,0.20195722579956055,0.003035998670384288,0.20195722579956055 +0.18488477170467377,0.001388245145790279,0.18488477170467377,25449.0,206,2025-12-15 11:20:40.816109,,,,,,,,, +0.17849349975585938,0.003042530035600066,0.17849349975585938,25499.0,206,2025-12-15 11:31:01.451833,,,,,,,,, +,,,,206,2025-12-15 11:43:30.122523,0.18909332156181335,0.0023090681061148643,0.18909332156181335,0.18268512189388275,0.0024032045621424913,0.18268512189388275,,, +,,,,206,2025-12-15 11:43:54.468751,,,,,,,0.19877494871616364,0.0031910156831145287,0.19877494871616364 +0.1771172434091568,0.0023776793386787176,0.1771172434091568,25549.0,207,2025-12-15 11:48:15.371727,,,,,,,,, +0.23438608646392822,0.0041619581170380116,0.23438608646392822,25599.0,207,2025-12-15 11:56:38.857826,,,,,,,,, +0.1909705400466919,0.002396219177171588,0.1909705400466919,25649.0,207,2025-12-15 12:07:12.889204,,,,,,,,, +,,,,207,2025-12-15 12:17:02.953857,0.19378043711185455,0.0031334368977695704,0.19378043711185455,0.18572242558002472,0.003835197538137436,0.18572242558002472,,, +,,,,207,2025-12-15 12:17:26.915591,,,,,,,0.19927990436553955,0.00277915527112782,0.19927990436553955 +0.1710750013589859,0.0024236042518168688,0.1710750013589859,25699.0,208,2025-12-15 12:28:21.671023,,,,,,,,, +0.19959859549999237,0.0018364350544288754,0.19959859549999237,25749.0,208,2025-12-15 12:41:38.943990,,,,,,,,, +,,,,208,2025-12-15 12:54:57.462596,0.18193286657333374,0.0018624747171998024,0.18193286657333374,0.17429593205451965,0.0025018833111971617,0.17429593205451965,,, +,,,,208,2025-12-15 12:55:44.299561,,,,,,,0.1988341212272644,0.002650184789672494,0.1988341212272644 +0.19342012703418732,0.003003517398610711,0.19342012703418732,25799.0,209,2025-12-15 13:01:11.947401,,,,,,,,, +0.24277541041374207,0.001451214193366468,0.24277541041374207,25849.0,209,2025-12-15 13:13:44.537413,,,,,,,,, +0.2155446708202362,0.005823667626827955,0.2155446708202362,25899.0,209,2025-12-15 13:24:26.222614,,,,,,,,, +,,,,209,2025-12-15 13:32:22.424547,0.1882031261920929,0.0026138497050851583,0.1882031261920929,0.17635004222393036,0.0029545703437179327,0.17635004222393036,,, +,,,,209,2025-12-15 13:32:46.087064,,,,,,,0.20093338191509247,0.002839765278622508,0.20093338191509247 +0.20150311291217804,0.0020586014725267887,0.20150311291217804,25949.0,210,2025-12-15 13:45:46.925782,,,,,,,,, +0.18458285927772522,0.002128985710442066,0.18458285927772522,25999.0,210,2025-12-15 13:55:34.724723,,,,,,,,, +,,,,210,2025-12-15 14:08:49.624733,0.18234364688396454,0.002502964809536934,0.18234364688396454,0.18577814102172852,0.0018814129289239645,0.18577814102172852,,, +,,,,210,2025-12-15 14:09:13.738652,,,,,,,0.19878432154655457,0.0033882050774991512,0.19878432154655457 +0.18905852735042572,0.0014387491391971707,0.18905852735042572,26049.0,211,2025-12-15 14:13:47.637784,,,,,,,,, +0.23481999337673187,0.0021229018457233906,0.23481999337673187,26099.0,211,2025-12-15 14:29:42.083644,,,,,,,,, +0.23038293421268463,0.002278164029121399,0.23038293421268463,26149.0,211,2025-12-15 14:40:51.786242,,,,,,,,, +,,,,211,2025-12-15 14:47:26.473528,0.1855478584766388,0.0022838448639959097,0.1855478584766388,0.18339966237545013,0.002759545110166073,0.18339966237545013,,, +,,,,211,2025-12-15 14:47:52.419174,,,,,,,0.20300281047821045,0.003146877745166421,0.20300281047821045 +0.24854901432991028,0.006352653261274099,0.24854901432991028,26199.0,212,2025-12-15 14:58:20.479332,,,,,,,,, +0.1939183473587036,0.002890868578106165,0.1939183473587036,26249.0,212,2025-12-15 15:10:23.197880,,,,,,,,, +,,,,212,2025-12-15 15:22:58.856314,0.1763271540403366,0.002776144538074732,0.1763271540403366,0.18033461272716522,0.002141491277143359,0.18033461272716522,,, +,,,,212,2025-12-15 15:23:23.303273,,,,,,,0.1977940946817398,0.0032023813109844923,0.1977940946817398 +0.18444395065307617,0.0013936104951426387,0.18444395065307617,26299.0,213,2025-12-15 15:28:46.781602,,,,,,,,, +0.15056796371936798,0.0013947610277682543,0.15056796371936798,26349.0,213,2025-12-15 15:39:41.705556,,,,,,,,, +0.16901279985904694,0.0011452987091615796,0.16901279985904694,26399.0,213,2025-12-15 15:52:49.217619,,,,,,,,, +,,,,213,2025-12-15 15:58:53.299341,0.18850885331630707,0.0027622641064226627,0.18850885331630707,0.17983053624629974,0.0023591986391693354,0.17983053624629974,,, +,,,,213,2025-12-15 15:59:17.433420,,,,,,,0.20071980357170105,0.0029885214753448963,0.20071980357170105 +0.17470937967300415,0.0021444656886160374,0.17470937967300415,26449.0,214,2025-12-15 16:09:50.065212,,,,,,,,, +0.19149580597877502,0.0010572628816589713,0.19149580597877502,26499.0,214,2025-12-15 16:24:49.872345,,,,,,,,, +,,,,214,2025-12-15 16:35:01.154484,0.18601882457733154,0.0023085337597876787,0.18601882457733154,0.18093828856945038,0.0048482040874660015,0.18093828856945038,,, +,,,,214,2025-12-15 16:35:25.638606,,,,,,,0.19721603393554688,0.0029839370399713516,0.19721603393554688 +0.20679038763046265,0.0019112733425572515,0.20679038763046265,26549.0,215,2025-12-15 16:40:49.546827,,,,,,,,, +0.19965003430843353,0.005518934223800898,0.19965003430843353,26599.0,215,2025-12-15 16:54:51.776587,,,,,,,,, +0.1546480506658554,0.001751866890117526,0.1546480506658554,26649.0,215,2025-12-15 17:10:16.037644,,,,,,,,, +,,,,215,2025-12-15 17:15:55.545535,0.17904457449913025,0.0019037978490814567,0.17904457449913025,0.19298067688941956,0.0038370105903595686,0.19298067688941956,,, +,,,,215,2025-12-15 17:16:20.432381,,,,,,,0.20066288113594055,0.0029759525787085295,0.20066288113594055 +0.1987965852022171,0.0022188490256667137,0.1987965852022171,26699.0,216,2025-12-15 17:30:23.892667,,,,,,,,, +0.2261037528514862,0.003626084653660655,0.2261037528514862,26749.0,216,2025-12-15 17:46:36.160890,,,,,,,,, +,,,,216,2025-12-15 17:59:13.726424,0.17968550324440002,0.0027255616150796413,0.17968550324440002,0.1818762570619583,0.0034380618017166853,0.1818762570619583,,, +,,,,216,2025-12-15 17:59:38.023273,,,,,,,0.19931627810001373,0.0031705901492387056,0.19931627810001373 +0.17743636667728424,0.004983116872608662,0.17743636667728424,26799.0,217,2025-12-15 18:07:14.991365,,,,,,,,, +0.2116614431142807,0.009479545056819916,0.2116614431142807,26849.0,217,2025-12-15 18:20:21.580011,,,,,,,,, +0.21088822185993195,0.0020748625975102186,0.21088822185993195,26899.0,217,2025-12-15 18:31:20.523199,,,,,,,,, +,,,,217,2025-12-15 18:36:59.973611,0.17933335900306702,0.0029114512726664543,0.17933335900306702,0.1866767704486847,0.003260800614953041,0.1866767704486847,,, +,,,,217,2025-12-15 18:37:25.285622,,,,,,,0.19659321010112762,0.003237929893657565,0.19659321010112762 +0.1890116035938263,0.001282867626287043,0.1890116035938263,26949.0,218,2025-12-15 18:48:23.753764,,,,,,,,, +0.22240565717220306,0.0027582054026424885,0.22240565717220306,26999.0,218,2025-12-15 18:59:54.741214,,,,,,,,, +,,,,218,2025-12-15 19:09:50.433272,0.18588969111442566,0.0021358213853091,0.18588969111442566,0.19015303254127502,0.0036992826499044895,0.19015303254127502,,, +,,,,218,2025-12-15 19:10:14.938014,,,,,,,0.19660413265228271,0.002779905218631029,0.19660413265228271 +0.22727292776107788,0.0022310898639261723,0.22727292776107788,27049.0,219,2025-12-15 19:16:25.093744,,,,,,,,, +0.18193639814853668,0.004770469851791859,0.18193639814853668,27099.0,219,2025-12-15 19:30:34.353014,,,,,,,,, +0.20583923161029816,0.0023615509271621704,0.20583923161029816,27149.0,219,2025-12-15 19:43:21.229488,,,,,,,,, +,,,,219,2025-12-15 19:48:29.702991,0.173540398478508,0.0017537210369482636,0.173540398478508,0.1881696730852127,0.0026929250452667475,0.1881696730852127,,, +,,,,219,2025-12-15 19:48:53.234930,,,,,,,0.2020132690668106,0.0028929824475198984,0.2020132690668106 +0.12076479196548462,0.0021981685422360897,0.12076479196548462,27199.0,220,2025-12-15 20:08:19.984585,,,,,,,,, +0.18918538093566895,0.0032367727253586054,0.18918538093566895,27249.0,220,2025-12-15 20:20:34.537361,,,,,,,,, +,,,,220,2025-12-15 20:31:26.971793,0.18653199076652527,0.0022040228359401226,0.18653199076652527,0.18179833889007568,0.002164409961551428,0.18179833889007568,,, +,,,,220,2025-12-15 20:31:52.610816,,,,,,,0.20253682136535645,0.003193130949512124,0.20253682136535645 +0.18921975791454315,0.0019699062686413527,0.18921975791454315,27299.0,221,2025-12-15 20:41:15.184833,,,,,,,,, +0.1873149573802948,0.007222316227853298,0.1873149573802948,27349.0,221,2025-12-15 20:52:28.307244,,,,,,,,, +0.1585211604833603,0.0015781046822667122,0.1585211604833603,27399.0,221,2025-12-15 21:04:38.081934,,,,,,,,, +,,,,221,2025-12-15 21:09:33.440024,0.1815909743309021,0.0031935011502355337,0.1815909743309021,0.17896775901317596,0.0038603281136602163,0.17896775901317596,,, +,,,,221,2025-12-15 21:09:58.047331,,,,,,,0.19772399961948395,0.003158371662721038,0.19772399961948395 +0.1376805603504181,0.0009724418050609529,0.1376805603504181,27449.0,222,2025-12-15 21:24:18.937784,,,,,,,,, +0.21094372868537903,0.004209873266518116,0.21094372868537903,27499.0,222,2025-12-15 21:33:19.757377,,,,,,,,, +,,,,222,2025-12-15 21:45:14.344231,0.18180440366268158,0.003083304036408663,0.18180440366268158,0.189732164144516,0.0037076002918183804,0.189732164144516,,, +,,,,222,2025-12-15 21:45:39.187524,,,,,,,0.19590383768081665,0.002869243500754237,0.19590383768081665 +0.2615472674369812,0.0033665867522358894,0.2615472674369812,27549.0,223,2025-12-15 21:55:42.614911,,,,,,,,, +0.22171099483966827,0.003015632973983884,0.22171099483966827,27599.0,223,2025-12-15 22:05:24.938459,,,,,,,,, +0.21172292530536652,0.008206081576645374,0.21172292530536652,27649.0,223,2025-12-15 22:14:23.193527,,,,,,,,, +,,,,223,2025-12-15 22:18:43.786130,0.18183428049087524,0.0022334959357976913,0.18183428049087524,0.1764533370733261,0.0026145048905164003,0.1764533370733261,,, +,,,,223,2025-12-15 22:19:07.142309,,,,,,,0.1964394748210907,0.002829818520694971,0.1964394748210907 +0.16795171797275543,0.0014298111200332642,0.16795171797275543,27699.0,224,2025-12-15 22:38:31.661495,,,,,,,,, +0.20640194416046143,0.0032834874000400305,0.20640194416046143,27749.0,224,2025-12-15 22:54:08.635370,,,,,,,,, +,,,,224,2025-12-15 23:05:47.859300,0.18127159774303436,0.0036176899448037148,0.18127159774303436,0.1860213428735733,0.002992416964843869,0.1860213428735733,,, +,,,,224,2025-12-15 23:06:11.335752,,,,,,,0.2002580612897873,0.002831299789249897,0.2002580612897873 +0.1654587686061859,0.004804700147360563,0.1654587686061859,27799.0,225,2025-12-15 23:13:57.495575,,,,,,,,, +0.19340774416923523,0.0017112726345658302,0.19340774416923523,27849.0,225,2025-12-15 23:24:52.675126,,,,,,,,, +0.1805659681558609,0.0035557139199227095,0.1805659681558609,27899.0,225,2025-12-15 23:37:12.548196,,,,,,,,, +,,,,225,2025-12-15 23:41:18.850442,0.18371233344078064,0.002512397477403283,0.18371233344078064,0.185917466878891,0.0025848925579339266,0.185917466878891,,, +,,,,225,2025-12-15 23:41:42.173365,,,,,,,0.20174455642700195,0.003002482233569026,0.20174455642700195 +0.2292415052652359,0.0019087533000856638,0.2292415052652359,27949.0,226,2025-12-15 23:53:19.659680,,,,,,,,, +0.24387089908123016,0.002533192979171872,0.24387089908123016,27999.0,226,2025-12-16 00:02:41.396339,,,,,,,,, +,,,,226,2025-12-16 00:14:14.766703,0.18910524249076843,0.003977570682764053,0.18910524249076843,0.18784743547439575,0.003319661831483245,0.18784743547439575,,, +,,,,226,2025-12-16 00:14:38.331807,,,,,,,0.20091885328292847,0.003153997240588069,0.20091885328292847 +0.1403442919254303,0.0011761944042518735,0.1403442919254303,28049.0,227,2025-12-16 00:22:55.803318,,,,,,,,, +0.14182329177856445,0.0009300172096118331,0.14182329177856445,28099.0,227,2025-12-16 00:38:20.792582,,,,,,,,, +,,,,227,2025-12-16 00:51:52.770459,0.18147580325603485,0.0033464052248746157,0.18147580325603485,0.18928378820419312,0.003916887100785971,0.18928378820419312,,, +,,,,227,2025-12-16 00:52:15.981735,,,,,,,0.19899417459964752,0.0031397014390677214,0.19899417459964752 +0.19116690754890442,0.006219581700861454,0.19116690754890442,28149.0,228,2025-12-16 00:54:23.279807,,,,,,,,, +0.21815142035484314,0.0027216544840484858,0.21815142035484314,28199.0,228,2025-12-16 01:07:08.357237,,,,,,,,, +0.17856429517269135,0.006336277816444635,0.17856429517269135,28249.0,228,2025-12-16 01:18:37.440867,,,,,,,,, +,,,,228,2025-12-16 01:28:41.507749,0.18182091414928436,0.0030918896663933992,0.18182091414928436,0.17717301845550537,0.002881903201341629,0.17717301845550537,,, +,,,,228,2025-12-16 01:29:06.183605,,,,,,,0.1955396980047226,0.0029901654925197363,0.1955396980047226 +0.21533608436584473,0.0032893591560423374,0.21533608436584473,28299.0,229,2025-12-16 01:40:03.074195,,,,,,,,, +0.21434231102466583,0.0017580384155735373,0.21434231102466583,28349.0,229,2025-12-16 01:52:45.272589,,,,,,,,, +,,,,229,2025-12-16 02:07:19.246046,0.1833161860704422,0.003545972751453519,0.1833161860704422,0.1848776787519455,0.0037492970004677773,0.1848776787519455,,, +,,,,229,2025-12-16 02:07:43.028769,,,,,,,0.19824758172035217,0.0031732721254229546,0.19824758172035217 +0.1471475511789322,0.0018934259423986077,0.1471475511789322,28399.0,230,2025-12-16 02:14:53.199346,,,,,,,,, +0.203581303358078,0.002737642964348197,0.203581303358078,28449.0,230,2025-12-16 02:30:44.796452,,,,,,,,, +0.26076915860176086,0.003583620535209775,0.26076915860176086,28499.0,230,2025-12-16 02:43:31.603771,,,,,,,,, +,,,,230,2025-12-16 02:50:52.481359,0.19027352333068848,0.002617950551211834,0.19027352333068848,0.1851169317960739,0.002537884283810854,0.1851169317960739,,, +,,,,230,2025-12-16 02:51:16.970857,,,,,,,0.19708755612373352,0.003115704283118248,0.19708755612373352 +0.2295180708169937,0.0013634838396683335,0.2295180708169937,28549.0,231,2025-12-16 03:02:47.281273,,,,,,,,, +0.17864367365837097,0.0018639119807630777,0.17864367365837097,28599.0,231,2025-12-16 03:13:39.738757,,,,,,,,, +,,,,231,2025-12-16 03:25:11.578191,0.18743321299552917,0.0039347438141703606,0.18743321299552917,0.1815708428621292,0.0030427316669374704,0.1815708428621292,,, +,,,,231,2025-12-16 03:25:35.778012,,,,,,,0.19643408060073853,0.003035815665498376,0.19643408060073853 +0.2221018224954605,0.0021052861120551825,0.2221018224954605,28649.0,232,2025-12-16 03:29:52.529716,,,,,,,,, +0.21707452833652496,0.0021270178258419037,0.21707452833652496,28699.0,232,2025-12-16 03:42:34.456332,,,,,,,,, +0.17664867639541626,0.0038641190622001886,0.17664867639541626,28749.0,232,2025-12-16 03:57:50.711085,,,,,,,,, +,,,,232,2025-12-16 04:04:51.140045,0.18349064886569977,0.002829959150403738,0.18349064886569977,0.18944565951824188,0.003287986619397998,0.18944565951824188,,, +,,,,232,2025-12-16 04:05:15.747081,,,,,,,0.19746775925159454,0.002754968823865056,0.19746775925159454 +0.2112460434436798,0.0017966729355975986,0.2112460434436798,28799.0,233,2025-12-16 04:18:58.495831,,,,,,,,, +0.1546412706375122,0.0014127425383776426,0.1546412706375122,28849.0,233,2025-12-16 04:29:07.631601,,,,,,,,, +,,,,233,2025-12-16 04:42:33.723741,0.1788129359483719,0.0033168173395097256,0.1788129359483719,0.18248488008975983,0.0033895843662321568,0.18248488008975983,,, +,,,,233,2025-12-16 04:42:58.289116,,,,,,,0.19910955429077148,0.002936127595603466,0.19910955429077148 +0.209840327501297,0.0031824270263314247,0.209840327501297,28899.0,234,2025-12-16 04:48:23.424495,,,,,,,,, +0.19373056292533875,0.001673859660513699,0.19373056292533875,28949.0,234,2025-12-16 05:00:52.604450,,,,,,,,, +0.20410855114459991,0.001681697554886341,0.20410855114459991,28999.0,234,2025-12-16 05:12:45.559513,,,,,,,,, +,,,,234,2025-12-16 05:19:26.133227,0.1765245795249939,0.002628414425998926,0.1765245795249939,0.18093734979629517,0.002950489055365324,0.18093734979629517,,, +,,,,234,2025-12-16 05:19:51.950135,,,,,,,0.201194167137146,0.0029976253863424063,0.201194167137146 +0.21944186091423035,0.0022623820696026087,0.21944186091423035,29049.0,235,2025-12-16 05:31:07.415387,,,,,,,,, +0.1847018152475357,0.0009343674755655229,0.1847018152475357,29099.0,235,2025-12-16 05:41:12.048001,,,,,,,,, +,,,,235,2025-12-16 05:54:23.769888,0.19493895769119263,0.003983409143984318,0.19493895769119263,0.18695934116840363,0.0022873487323522568,0.18695934116840363,,, +,,,,235,2025-12-16 05:54:47.930729,,,,,,,0.19532768428325653,0.0028355324175208807,0.19532768428325653 +0.2309790402650833,0.006500988267362118,0.2309790402650833,29149.0,236,2025-12-16 05:59:52.592485,,,,,,,,, +0.23012417554855347,0.0025114156305789948,0.23012417554855347,29199.0,236,2025-12-16 06:12:42.781302,,,,,,,,, +0.20917464792728424,0.0014113386860117316,0.20917464792728424,29249.0,236,2025-12-16 06:21:04.111634,,,,,,,,, +,,,,236,2025-12-16 06:28:46.025688,0.18581648170948029,0.0027329253498464823,0.18581648170948029,0.18406103551387787,0.002178841270506382,0.18406103551387787,,, +,,,,236,2025-12-16 06:29:10.003815,,,,,,,0.19803063571453094,0.002823172602802515,0.19803063571453094 +0.21085777878761292,0.0012851983774453402,0.21085777878761292,29299.0,237,2025-12-16 06:39:12.386900,,,,,,,,, +0.19328151643276215,0.001957177184522152,0.19328151643276215,29349.0,237,2025-12-16 06:52:32.158895,,,,,,,,, +,,,,237,2025-12-16 07:09:02.167691,0.1762024313211441,0.002687338972464204,0.1762024313211441,0.17883440852165222,0.0014757312601432204,0.17883440852165222,,, +,,,,237,2025-12-16 07:09:26.974095,,,,,,,0.20024482905864716,0.0031769999768584967,0.20024482905864716 +0.2027127891778946,0.002808402990922332,0.2027127891778946,29399.0,238,2025-12-16 07:22:31.601231,,,,,,,,, +0.16732747852802277,0.004808586556464434,0.16732747852802277,29449.0,238,2025-12-16 07:34:59.407903,,,,,,,,, +0.17681413888931274,0.0018821139819920063,0.17681413888931274,29499.0,238,2025-12-16 07:45:04.370037,,,,,,,,, +,,,,238,2025-12-16 07:51:22.078694,0.18336503207683563,0.002835116581991315,0.18336503207683563,0.18825842440128326,0.003417189000174403,0.18825842440128326,,, +,,,,238,2025-12-16 07:51:46.301488,,,,,,,0.1974838227033615,0.0029388421680778265,0.1974838227033615 +0.1919972449541092,0.002588890492916107,0.1919972449541092,29549.0,239,2025-12-16 08:04:07.726367,,,,,,,,, +0.17926879227161407,0.002921657171100378,0.17926879227161407,29599.0,239,2025-12-16 08:14:36.756769,,,,,,,,, +,,,,239,2025-12-16 08:25:14.975232,0.18873924016952515,0.003057805821299553,0.18873924016952515,0.18409574031829834,0.004260390996932983,0.18409574031829834,,, +,,,,239,2025-12-16 08:25:39.028614,,,,,,,0.1950123906135559,0.0030449663754552603,0.1950123906135559 +0.17655035853385925,0.0012954863486811519,0.17655035853385925,29649.0,240,2025-12-16 08:33:40.747448,,,,,,,,, +0.2278309464454651,0.003469983348622918,0.2278309464454651,29699.0,240,2025-12-16 08:46:01.346462,,,,,,,,, +0.23484458029270172,0.005211300682276487,0.23484458029270172,29749.0,240,2025-12-16 08:56:17.151656,,,,,,,,, +,,,,240,2025-12-16 09:02:04.556182,0.18310344219207764,0.002433511894196272,0.18310344219207764,0.1801697164773941,0.0022172241006046534,0.1801697164773941,,, +,,,,240,2025-12-16 09:02:30.083933,,,,,,,0.19749848544597626,0.003402393776923418,0.19749848544597626 +0.20337477326393127,0.003493475029245019,0.20337477326393127,29799.0,241,2025-12-16 09:15:50.551782,,,,,,,,, +0.16998638212680817,0.0017220425652340055,0.16998638212680817,29849.0,241,2025-12-16 09:28:23.472965,,,,,,,,, +,,,,241,2025-12-16 09:38:41.679733,0.17328472435474396,0.0016639402601867914,0.17328472435474396,0.18155600130558014,0.0025759171694517136,0.18155600130558014,,, +,,,,241,2025-12-16 09:39:05.860922,,,,,,,0.1964471936225891,0.0028280229307711124,0.1964471936225891 +0.18911482393741608,0.0022599948570132256,0.18911482393741608,29899.0,242,2025-12-16 09:48:45.776402,,,,,,,,, +0.18325795233249664,0.005964396521449089,0.18325795233249664,29949.0,242,2025-12-16 09:59:54.575194,,,,,,,,, +0.19609230756759644,0.00129069690592587,0.19609230756759644,29999.0,242,2025-12-16 10:16:27.191109,,,,,,,,, +,,,,242,2025-12-16 10:22:01.093084,0.1864643394947052,0.0033968130592256784,0.1864643394947052,0.1729830652475357,0.0020164656452834606,0.1729830652475357,,, +,,,,242,2025-12-16 10:22:47.916403,,,,,,,0.19673927128314972,0.003065993543714285,0.19673927128314972 +0.2417839914560318,0.005931031424552202,0.2417839914560318,30049.0,243,2025-12-16 10:35:05.996170,,,,,,,,, +0.20275583863258362,0.006820057984441519,0.20275583863258362,30099.0,243,2025-12-16 10:48:33.565549,,,,,,,,, +,,,,243,2025-12-16 10:57:44.090514,0.18553447723388672,0.0021400239784270525,0.18553447723388672,0.17405056953430176,0.0028295305091887712,0.17405056953430176,,, +,,,,243,2025-12-16 10:58:08.027666,,,,,,,0.19993408024311066,0.00322366482578218,0.19993408024311066 +0.18688878417015076,0.001566243590787053,0.18688878417015076,30149.0,244,2025-12-16 11:07:40.955288,,,,,,,,, +0.23058423399925232,0.0034997225739061832,0.23058423399925232,30199.0,244,2025-12-16 11:18:12.190405,,,,,,,,, +0.17187166213989258,0.003113693790510297,0.17187166213989258,30249.0,244,2025-12-16 11:29:44.625333,,,,,,,,, +,,,,244,2025-12-16 11:34:45.330840,0.19092927873134613,0.0032661219593137503,0.19092927873134613,0.1790200173854828,0.0038315383717417717,0.1790200173854828,,, +,,,,244,2025-12-16 11:35:08.505848,,,,,,,0.19271545112133026,0.0027578449808061123,0.19271545112133026 +0.21095149219036102,0.005385282449424267,0.21095149219036102,30299.0,245,2025-12-16 11:49:34.813398,,,,,,,,, +0.2410937249660492,0.002469825092703104,0.2410937249660492,30349.0,245,2025-12-16 12:00:20.767592,,,,,,,,, +,,,,245,2025-12-16 12:09:38.892312,0.17909792065620422,0.002526328433305025,0.17909792065620422,0.18206916749477386,0.002482213079929352,0.18206916749477386,,, +,,,,245,2025-12-16 12:10:04.245610,,,,,,,0.19641253352165222,0.002876757178455591,0.19641253352165222 +0.15794838964939117,0.0008850986487232149,0.15794838964939117,30399.0,246,2025-12-16 12:16:24.506157,,,,,,,,, +0.18395887315273285,0.0022192467004060745,0.18395887315273285,30449.0,246,2025-12-16 12:29:40.181083,,,,,,,,, +0.17345640063285828,0.002452685497701168,0.17345640063285828,30499.0,246,2025-12-16 12:39:19.303007,,,,,,,,, +,,,,246,2025-12-16 12:44:00.319794,0.18651019036769867,0.0030282405205070972,0.18651019036769867,0.18750956654548645,0.003079809248447418,0.18750956654548645,,, +,,,,246,2025-12-16 12:44:23.948932,,,,,,,0.19870977103710175,0.003126712515950203,0.19870977103710175 +0.20351572334766388,0.004651546943932772,0.20351572334766388,30549.0,247,2025-12-16 12:59:09.642331,,,,,,,,, +0.2098948210477829,0.0016542556695640087,0.2098948210477829,30599.0,247,2025-12-16 13:09:56.136769,,,,,,,,, +,,,,247,2025-12-16 13:21:39.528927,0.18013139069080353,0.003123106202110648,0.18013139069080353,0.18124817311763763,0.003244492458179593,0.18124817311763763,,, +,,,,247,2025-12-16 13:22:03.768967,,,,,,,0.19647541642189026,0.002859334461390972,0.19647541642189026 +0.20237578451633453,0.006666473113000393,0.20237578451633453,30649.0,248,2025-12-16 13:30:30.546346,,,,,,,,, +0.1452348679304123,0.0010644999565556645,0.1452348679304123,30699.0,248,2025-12-16 13:43:23.819307,,,,,,,,, +0.1818534880876541,0.0024436847306787968,0.1818534880876541,30749.0,248,2025-12-16 13:55:15.043002,,,,,,,,, +,,,,248,2025-12-16 13:59:43.935709,0.18464702367782593,0.0034059772733598948,0.18464702367782593,0.17812088131904602,0.0025321512948721647,0.17812088131904602,,, +,,,,248,2025-12-16 14:00:08.367406,,,,,,,0.19242188334465027,0.0028567747212946415,0.19242188334465027 +0.23002801835536957,0.001745465095154941,0.23002801835536957,30799.0,249,2025-12-16 14:17:01.256907,,,,,,,,, +0.22234578430652618,0.0034374319948256016,0.22234578430652618,30849.0,249,2025-12-16 14:29:21.389765,,,,,,,,, +,,,,249,2025-12-16 14:40:46.253184,0.18780174851417542,0.0029644181486219168,0.18780174851417542,0.19724591076374054,0.0030913909431546926,0.19724591076374054,,, +,,,,249,2025-12-16 14:41:11.336156,,,,,,,0.19535957276821136,0.003168632509186864,0.19535957276821136 +0.20733018219470978,0.0015724507393315434,0.20733018219470978,30899.0,250,2025-12-16 14:47:44.662859,,,,,,,,, +0.199334055185318,0.005114336963742971,0.199334055185318,30949.0,250,2025-12-16 15:00:11.225833,,,,,,,,, +0.17277821898460388,0.0013667643070220947,0.17277821898460388,30999.0,250,2025-12-16 15:09:22.777331,,,,,,,,, +,,,,250,2025-12-16 15:13:25.203798,0.1792389303445816,0.003387100761756301,0.1792389303445816,0.18077747523784637,0.002393915317952633,0.18077747523784637,,, +,,,,250,2025-12-16 15:13:50.422560,,,,,,,0.19700615108013153,0.00319044035859406,0.19700615108013153 +0.22203132510185242,0.0017165992176160216,0.22203132510185242,31049.0,251,2025-12-16 15:26:43.090306,,,,,,,,, +0.23567241430282593,0.0018863603472709656,0.23567241430282593,31099.0,251,2025-12-16 15:38:26.053441,,,,,,,,, +,,,,251,2025-12-16 15:47:17.550360,0.1794758290052414,0.0029077641665935516,0.1794758290052414,0.17597456276416779,0.0027902701403945684,0.17597456276416779,,, +,,,,251,2025-12-16 15:47:42.611754,,,,,,,0.1976393610239029,0.0029469483997672796,0.1976393610239029 +0.18370170891284943,0.001713149482384324,0.18370170891284943,31149.0,252,2025-12-16 15:57:03.279610,,,,,,,,, +0.17853759229183197,0.004629850387573242,0.17853759229183197,31199.0,252,2025-12-16 16:06:25.408683,,,,,,,,, +,,,,252,2025-12-16 16:22:02.129320,0.18073517084121704,0.0026961350813508034,0.18073517084121704,0.17961369454860687,0.0031534836161881685,0.17961369454860687,,, +,,,,252,2025-12-16 16:22:27.155152,,,,,,,0.1959397941827774,0.002957369899377227,0.1959397941827774 +0.24215905368328094,0.0015214458107948303,0.24215905368328094,31249.0,253,2025-12-16 16:25:06.936954,,,,,,,,, +0.16523303091526031,0.0038586219307035208,0.16523303091526031,31299.0,253,2025-12-16 16:37:05.564929,,,,,,,,, +0.18365801870822906,0.002161671407520771,0.18365801870822906,31349.0,253,2025-12-16 16:47:26.714435,,,,,,,,, +,,,,253,2025-12-16 16:55:38.082345,0.17711730301380157,0.004119785502552986,0.17711730301380157,0.17951470613479614,0.003442163346335292,0.17951470613479614,,, +,,,,253,2025-12-16 16:56:02.236886,,,,,,,0.19397765398025513,0.0030436438973993063,0.19397765398025513 +0.1773979514837265,0.001564479200169444,0.1773979514837265,31399.0,254,2025-12-16 17:05:29.290463,,,,,,,,, +0.18966318666934967,0.003808417823165655,0.18966318666934967,31449.0,254,2025-12-16 17:18:30.395733,,,,,,,,, +,,,,254,2025-12-16 17:31:54.762363,0.19247065484523773,0.0026445600669831038,0.19247065484523773,0.1779184341430664,0.002158108400180936,0.1779184341430664,,, +,,,,254,2025-12-16 17:32:18.681587,,,,,,,0.19808509945869446,0.0031042187474668026,0.19808509945869446 +0.19416725635528564,0.005056817550212145,0.19416725635528564,31499.0,255,2025-12-16 17:35:35.524803,,,,,,,,, +0.17738448083400726,0.0012819190742447972,0.17738448083400726,31549.0,255,2025-12-16 17:48:55.354873,,,,,,,,, +0.20728915929794312,0.0010359515435993671,0.20728915929794312,31599.0,255,2025-12-16 18:04:12.383007,,,,,,,,, +,,,,255,2025-12-16 18:12:01.700050,0.18207742273807526,0.002507682889699936,0.18207742273807526,0.18499071896076202,0.005223874934017658,0.18499071896076202,,, +,,,,255,2025-12-16 18:12:24.543382,,,,,,,0.19646111130714417,0.0032388558611273766,0.19646111130714417 +0.191259503364563,0.0017913085175678134,0.191259503364563,31649.0,256,2025-12-16 18:21:10.334428,,,,,,,,, +0.14900615811347961,0.001342513132840395,0.14900615811347961,31699.0,256,2025-12-16 18:32:07.480393,,,,,,,,, +,,,,256,2025-12-16 18:44:57.163652,0.18994642794132233,0.004003677982836962,0.18994642794132233,0.19461531937122345,0.0023544791620224714,0.19461531937122345,,, +,,,,256,2025-12-16 18:45:21.538916,,,,,,,0.19630885124206543,0.002745602745562792,0.19630885124206543 +0.22017724812030792,0.0020154668018221855,0.22017724812030792,31749.0,257,2025-12-16 18:49:57.603295,,,,,,,,, +0.18499964475631714,0.0021607568487524986,0.18499964475631714,31799.0,257,2025-12-16 19:03:04.940680,,,,,,,,, +0.18281006813049316,0.0013246372109279037,0.18281006813049316,31849.0,257,2025-12-16 19:13:22.079683,,,,,,,,, +,,,,257,2025-12-16 19:21:52.095697,0.1790338009595871,0.002692840527743101,0.1790338009595871,0.1845400631427765,0.0024829141329973936,0.1845400631427765,,, +,,,,257,2025-12-16 19:22:17.299990,,,,,,,0.19781862199306488,0.002840863773599267,0.19781862199306488 +0.2383977621793747,0.0067029413767158985,0.2383977621793747,31899.0,258,2025-12-16 19:32:46.380083,,,,,,,,, +0.2388399988412857,0.0027516107074916363,0.2388399988412857,31949.0,258,2025-12-16 19:50:25.126555,,,,,,,,, +,,,,258,2025-12-16 20:03:37.927020,0.17119143903255463,0.002924197819083929,0.17119143903255463,0.17800462245941162,0.0025579233188182116,0.17800462245941162,,, +,,,,258,2025-12-16 20:04:02.518758,,,,,,,0.1995142698287964,0.003280536038801074,0.1995142698287964 +0.24021616578102112,0.00805174931883812,0.24021616578102112,31999.0,259,2025-12-16 20:08:03.048340,,,,,,,,, +0.1992270052433014,0.005818352568894625,0.1992270052433014,32049.0,259,2025-12-16 20:20:25.017139,,,,,,,,, +0.17922690510749817,0.002064632950350642,0.17922690510749817,32099.0,259,2025-12-16 20:30:34.132671,,,,,,,,, +,,,,259,2025-12-16 20:41:27.381485,0.18168389797210693,0.0035349195823073387,0.18168389797210693,0.175323948264122,0.002655167831107974,0.175323948264122,,, +,,,,259,2025-12-16 20:41:52.936234,,,,,,,0.19721762835979462,0.0032055117189884186,0.19721762835979462 +0.16894812881946564,0.0016993725439533591,0.16894812881946564,32149.0,260,2025-12-16 20:54:06.664848,,,,,,,,, +0.2016793191432953,0.002027052454650402,0.2016793191432953,32199.0,260,2025-12-16 21:09:09.842893,,,,,,,,, +,,,,260,2025-12-16 21:20:18.705619,0.17777420580387115,0.001743022003211081,0.17777420580387115,0.1864989846944809,0.0025141946971416473,0.1864989846944809,,, +,,,,260,2025-12-16 21:20:43.509913,,,,,,,0.20200516283512115,0.003591702086851001,0.20200516283512115 +0.20974425971508026,0.001908661681227386,0.20974425971508026,32249.0,261,2025-12-16 21:24:44.696192,,,,,,,,, +0.19019556045532227,0.001622206182219088,0.19019556045532227,32299.0,261,2025-12-16 21:38:03.460047,,,,,,,,, +0.16311174631118774,0.0013742445735260844,0.16311174631118774,32349.0,261,2025-12-16 21:50:42.556915,,,,,,,,, +,,,,261,2025-12-16 21:57:07.261349,0.1831309050321579,0.002623150125145912,0.1831309050321579,0.17836536467075348,0.0017292186385020614,0.17836536467075348,,, +,,,,261,2025-12-16 21:57:31.434707,,,,,,,0.19655707478523254,0.0029079769738018513,0.19655707478523254 +0.22123445570468903,0.0018405921291559935,0.22123445570468903,32399.0,262,2025-12-16 22:07:47.965024,,,,,,,,, +0.23602479696273804,0.0018994661513715982,0.23602479696273804,32449.0,262,2025-12-16 22:19:56.669345,,,,,,,,, +,,,,262,2025-12-16 22:30:17.195517,0.18302398920059204,0.003141951048746705,0.18302398920059204,0.18242645263671875,0.0023049490991979837,0.18242645263671875,,, +,,,,262,2025-12-16 22:30:42.279888,,,,,,,0.1986987292766571,0.0029592039063572884,0.1986987292766571 +0.18567581474781036,0.00446312315762043,0.18567581474781036,32499.0,263,2025-12-16 22:38:44.377990,,,,,,,,, +0.18673546612262726,0.0022020682226866484,0.18673546612262726,32549.0,263,2025-12-16 22:50:33.307877,,,,,,,,, +0.21326591074466705,0.005776406731456518,0.21326591074466705,32599.0,263,2025-12-16 23:03:59.828529,,,,,,,,, +,,,,263,2025-12-16 23:10:57.485513,0.1860974133014679,0.002933477284386754,0.1860974133014679,0.18040189146995544,0.0027711910661309958,0.18040189146995544,,, +,,,,263,2025-12-16 23:11:21.478936,,,,,,,0.19484813511371613,0.0027145398780703545,0.19484813511371613 +0.18594205379486084,0.002131484914571047,0.18594205379486084,32649.0,264,2025-12-16 23:25:34.215526,,,,,,,,, +0.17505015432834625,0.0018688510172069073,0.17505015432834625,32699.0,264,2025-12-16 23:44:49.253566,,,,,,,,, +,,,,264,2025-12-16 23:57:42.130245,0.17961998283863068,0.0023051914758980274,0.17961998283863068,0.17824895679950714,0.002808269113302231,0.17824895679950714,,, +,,,,264,2025-12-16 23:58:06.114531,,,,,,,0.19923706352710724,0.003055788343772292,0.19923706352710724 +0.2092004120349884,0.0030635541770607233,0.2092004120349884,32749.0,265,2025-12-17 00:02:58.962419,,,,,,,,, +0.21140439808368683,0.002139734337106347,0.21140439808368683,32799.0,265,2025-12-17 00:14:46.439745,,,,,,,,, +0.22980254888534546,0.005662687122821808,0.22980254888534546,32849.0,265,2025-12-17 00:28:41.077658,,,,,,,,, +,,,,265,2025-12-17 00:34:56.225696,0.18629761040210724,0.004274560604244471,0.18629761040210724,0.17390047013759613,0.0032715590205043554,0.17390047013759613,,, +,,,,265,2025-12-17 00:35:21.336485,,,,,,,0.19424143433570862,0.0033134210389107466,0.19424143433570862 +0.18999795615673065,0.0019932682625949383,0.18999795615673065,32899.0,266,2025-12-17 00:50:03.540108,,,,,,,,, +0.20864006876945496,0.0022052587009966373,0.20864006876945496,32949.0,266,2025-12-17 00:58:18.111237,,,,,,,,, +,,,,266,2025-12-17 01:12:06.952146,0.18616285920143127,0.0032371534034609795,0.18616285920143127,0.18307939171791077,0.0025696090888231993,0.18307939171791077,,, +,,,,266,2025-12-17 01:12:31.444304,,,,,,,0.19482466578483582,0.002718135481700301,0.19482466578483582 +0.1745367795228958,0.0014497512020170689,0.1745367795228958,32999.0,267,2025-12-17 01:23:15.140532,,,,,,,,, +0.16906918585300446,0.001471291994675994,0.16906918585300446,33049.0,267,2025-12-17 01:35:51.622565,,,,,,,,, +0.19098064303398132,0.001495660631917417,0.19098064303398132,33099.0,267,2025-12-17 01:44:34.848651,,,,,,,,, +,,,,267,2025-12-17 01:51:40.982591,0.17668205499649048,0.0020466006826609373,0.17668205499649048,0.17855453491210938,0.003862554905936122,0.17855453491210938,,, +,,,,267,2025-12-17 01:52:05.404239,,,,,,,0.19664451479911804,0.003040928393602371,0.19664451479911804 +0.1913306564092636,0.0025951038114726543,0.1913306564092636,33149.0,268,2025-12-17 02:02:53.236464,,,,,,,,, +0.2229653000831604,0.004564368166029453,0.2229653000831604,33199.0,268,2025-12-17 02:13:10.831010,,,,,,,,, +,,,,268,2025-12-17 02:26:16.251805,0.18154190480709076,0.0027749978471547365,0.18154190480709076,0.18422673642635345,0.0021657978650182486,0.18422673642635345,,, +,,,,268,2025-12-17 02:26:41.672498,,,,,,,0.19368478655815125,0.002752013271674514,0.19368478655815125 +0.2058575302362442,0.0022402347531169653,0.2058575302362442,33249.0,269,2025-12-17 02:33:00.302954,,,,,,,,, +0.1982947736978531,0.002440057694911957,0.1982947736978531,33299.0,269,2025-12-17 02:45:26.205901,,,,,,,,, +0.21919234097003937,0.002394532086327672,0.21919234097003937,33349.0,269,2025-12-17 02:58:04.627661,,,,,,,,, +,,,,269,2025-12-17 03:03:08.948884,0.17924724519252777,0.002012401819229126,0.17924724519252777,0.18710613250732422,0.0030461703427135944,0.18710613250732422,,, +,,,,269,2025-12-17 03:03:33.481570,,,,,,,0.19475485384464264,0.002867935225367546,0.19475485384464264 +0.19090700149536133,0.0025018847081810236,0.19090700149536133,33399.0,270,2025-12-17 03:17:44.219580,,,,,,,,, +0.2105451226234436,0.0052385153248906136,0.2105451226234436,33449.0,270,2025-12-17 03:28:33.369557,,,,,,,,, +,,,,270,2025-12-17 03:41:36.109725,0.17997345328330994,0.003063084790483117,0.17997345328330994,0.1825525164604187,0.004257783759385347,0.1825525164604187,,, +,,,,270,2025-12-17 03:42:00.846746,,,,,,,0.19738933444023132,0.003107775002717972,0.19738933444023132 +0.2147376984357834,0.00320486631244421,0.2147376984357834,33499.0,271,2025-12-17 03:47:53.201671,,,,,,,,, +0.21739870309829712,0.00196065753698349,0.21739870309829712,33549.0,271,2025-12-17 03:57:31.366055,,,,,,,,, +0.177575945854187,0.0011410029837861657,0.177575945854187,33599.0,271,2025-12-17 04:16:44.501611,,,,,,,,, +,,,,271,2025-12-17 04:21:29.607554,0.17563340067863464,0.002852171426638961,0.17563340067863464,0.17902012169361115,0.002315330784767866,0.17902012169361115,,, +,,,,271,2025-12-17 04:21:55.316180,,,,,,,0.19431880116462708,0.002734204987064004,0.19431880116462708 +0.18823955953121185,0.0016654656501486897,0.18823955953121185,33649.0,272,2025-12-17 04:35:20.688683,,,,,,,,, +0.22261209785938263,0.004264821298420429,0.22261209785938263,33699.0,272,2025-12-17 04:44:28.491551,,,,,,,,, +,,,,272,2025-12-17 04:54:09.602301,0.18707282841205597,0.00314976554363966,0.18707282841205597,0.17311924695968628,0.002870722906664014,0.17311924695968628,,, +,,,,272,2025-12-17 04:54:56.730889,,,,,,,0.19680994749069214,0.0028877651784569025,0.19680994749069214 +0.20985500514507294,0.005368922837078571,0.20985500514507294,33749.0,273,2025-12-17 05:06:53.760666,,,,,,,,, +0.1960938423871994,0.00454776780679822,0.1960938423871994,33799.0,273,2025-12-17 05:17:28.419841,,,,,,,,, +0.1862405687570572,0.00758683355525136,0.1862405687570572,33849.0,273,2025-12-17 05:28:12.228264,,,,,,,,, +,,,,273,2025-12-17 05:32:40.766661,0.1766050010919571,0.003040880663320422,0.1766050010919571,0.17493464052677155,0.002804349409416318,0.17493464052677155,,, +,,,,273,2025-12-17 05:33:04.644722,,,,,,,0.19620729982852936,0.0030506926123052835,0.19620729982852936 +0.19487231969833374,0.0011071480112150311,0.19487231969833374,33899.0,274,2025-12-17 05:44:32.129306,,,,,,,,, +0.2386454939842224,0.004690142348408699,0.2386454939842224,33949.0,274,2025-12-17 06:05:26.766311,,,,,,,,, +,,,,274,2025-12-17 06:14:59.612958,0.1797480434179306,0.0033584388438612223,0.1797480434179306,0.17987410724163055,0.0025378158316016197,0.17987410724163055,,, +,,,,274,2025-12-17 06:15:25.200658,,,,,,,0.19587121903896332,0.0028579176869243383,0.19587121903896332 +0.1734086126089096,0.0014764826046302915,0.1734086126089096,33999.0,275,2025-12-17 06:25:51.644061,,,,,,,,, +0.18508633971214294,0.005945656914263964,0.18508633971214294,34049.0,275,2025-12-17 06:40:14.557155,,,,,,,,, +0.18939648568630219,0.0019318374106660485,0.18939648568630219,34099.0,275,2025-12-17 06:49:37.992750,,,,,,,,, +,,,,275,2025-12-17 06:53:57.063868,0.18250633776187897,0.0029234413523226976,0.18250633776187897,0.1786859929561615,0.002937258454039693,0.1786859929561615,,, +,,,,275,2025-12-17 06:54:21.079546,,,,,,,0.19528710842132568,0.0030083402525633574,0.19528710842132568 +0.1720307320356369,0.004921875428408384,0.1720307320356369,34149.0,276,2025-12-17 07:08:21.190429,,,,,,,,, +0.1856003999710083,0.0015132151311263442,0.1856003999710083,34199.0,276,2025-12-17 07:24:18.211979,,,,,,,,, +,,,,276,2025-12-17 07:32:30.324588,0.18550436198711395,0.003909315448254347,0.18550436198711395,0.18410661816596985,0.002489775186404586,0.18410661816596985,,, +,,,,276,2025-12-17 07:32:55.933639,,,,,,,0.19784851372241974,0.0031336178071796894,0.19784851372241974 +0.19376933574676514,0.004961185157299042,0.19376933574676514,34249.0,277,2025-12-17 07:42:06.520574,,,,,,,,, +0.19169874489307404,0.0011488148011267185,0.19169874489307404,34299.0,277,2025-12-17 07:55:58.314212,,,,,,,,, +,,,,277,2025-12-17 08:11:32.285518,0.17931777238845825,0.0025082523934543133,0.17931777238845825,0.18325208127498627,0.002775000873953104,0.18325208127498627,,, +,,,,277,2025-12-17 08:11:56.504148,,,,,,,0.20205138623714447,0.0034028554800897837,0.20205138623714447 +0.2142895609140396,0.001716902945190668,0.2142895609140396,34349.0,278,2025-12-17 08:14:51.202118,,,,,,,,, +0.209428071975708,0.00517966179177165,0.209428071975708,34399.0,278,2025-12-17 08:24:01.526047,,,,,,,,, +0.171523779630661,0.0038607935421168804,0.171523779630661,34449.0,278,2025-12-17 08:34:35.325804,,,,,,,,, +,,,,278,2025-12-17 08:45:32.416611,0.17269670963287354,0.002637700643390417,0.17269670963287354,0.18328173458576202,0.0023927579168230295,0.18328173458576202,,, +,,,,278,2025-12-17 08:45:58.217285,,,,,,,0.19853775203227997,0.0031315379310399294,0.19853775203227997 +0.18562062084674835,0.0016185209387913346,0.18562062084674835,34499.0,279,2025-12-17 08:55:18.507965,,,,,,,,, +0.2361741065979004,0.005059600807726383,0.2361741065979004,34549.0,279,2025-12-17 09:04:38.612783,,,,,,,,, +,,,,279,2025-12-17 09:23:03.532129,0.1778113842010498,0.0028631638269871473,0.1778113842010498,0.1809278279542923,0.0027123959735035896,0.1809278279542923,,, +,,,,279,2025-12-17 09:23:26.964275,,,,,,,0.19861376285552979,0.002972915070131421,0.19861376285552979 +0.19146965444087982,0.001468624803237617,0.19146965444087982,34599.0,280,2025-12-17 09:28:00.120736,,,,,,,,, +0.2551257908344269,0.0043366155587136745,0.2551257908344269,34649.0,280,2025-12-17 09:41:38.436435,,,,,,,,, +0.20107844471931458,0.002793736057356,0.20107844471931458,34699.0,280,2025-12-17 09:57:45.266602,,,,,,,,, +,,,,280,2025-12-17 10:05:18.212772,0.1821620762348175,0.0022026398219168186,0.1821620762348175,0.18346497416496277,0.0027477333787828684,0.18346497416496277,,, +,,,,280,2025-12-17 10:05:41.333154,,,,,,,0.19545620679855347,0.002794065745547414,0.19545620679855347 +0.2082589566707611,0.0033366642892360687,0.2082589566707611,34749.0,281,2025-12-17 10:17:52.963932,,,,,,,,, +0.20061784982681274,0.001522244536317885,0.20061784982681274,34799.0,281,2025-12-17 10:32:45.919299,,,,,,,,, +,,,,281,2025-12-17 10:45:12.654189,0.17905855178833008,0.002224571770057082,0.17905855178833008,0.18500962853431702,0.0031305919401347637,0.18500962853431702,,, +,,,,281,2025-12-17 10:45:36.910280,,,,,,,0.19614730775356293,0.0028053265996277332,0.19614730775356293 +0.21815598011016846,0.0023282053880393505,0.21815598011016846,34849.0,282,2025-12-17 10:50:37.756794,,,,,,,,, +0.21717560291290283,0.002929470269009471,0.21717560291290283,34899.0,282,2025-12-17 11:03:42.513180,,,,,,,,, +0.19139830768108368,0.005905340425670147,0.19139830768108368,34949.0,282,2025-12-17 11:13:07.970571,,,,,,,,, +,,,,282,2025-12-17 11:25:47.862956,0.18543066084384918,0.0032872832380235195,0.18543066084384918,0.18540838360786438,0.002622682601213455,0.18540838360786438,,, +,,,,282,2025-12-17 11:26:12.125322,,,,,,,0.19573551416397095,0.003163708373904228,0.19573551416397095 +0.2070024460554123,0.0052359807305037975,0.2070024460554123,34999.0,283,2025-12-17 11:36:24.058976,,,,,,,,, +0.21377873420715332,0.0037270679604262114,0.21377873420715332,35049.0,283,2025-12-17 11:51:13.939181,,,,,,,,, +,,,,283,2025-12-17 12:02:43.820348,0.17973951995372772,0.0035935610067099333,0.17973951995372772,0.1859530508518219,0.0023296200670301914,0.1859530508518219,,, +,,,,283,2025-12-17 12:03:08.370487,,,,,,,0.19583484530448914,0.002952990820631385,0.19583484530448914 +0.2094549536705017,0.0035636955872178078,0.2094549536705017,35099.0,284,2025-12-17 12:06:44.652625,,,,,,,,, +0.1801379919052124,0.0013365230988711119,0.1801379919052124,35149.0,284,2025-12-17 12:23:04.714279,,,,,,,,, +0.2103913277387619,0.002710232511162758,0.2103913277387619,35199.0,284,2025-12-17 12:34:44.309399,,,,,,,,, +,,,,284,2025-12-17 12:43:39.414536,0.18591295182704926,0.002684756414964795,0.18591295182704926,0.17847716808319092,0.002506059827283025,0.17847716808319092,,, +,,,,284,2025-12-17 12:44:03.223675,,,,,,,0.19588665664196014,0.0031442022882401943,0.19588665664196014 +0.21185864508152008,0.002574816346168518,0.21185864508152008,35249.0,285,2025-12-17 12:56:42.649803,,,,,,,,, +0.22073468565940857,0.002180121373385191,0.22073468565940857,35299.0,285,2025-12-17 13:08:17.433776,,,,,,,,, +,,,,285,2025-12-17 13:21:15.621192,0.17108570039272308,0.0021924800239503384,0.17108570039272308,0.18105027079582214,0.003650401486083865,0.18105027079582214,,, +,,,,285,2025-12-17 13:21:39.628510,,,,,,,0.19681060314178467,0.0031003348995000124,0.19681060314178467 +0.19068877398967743,0.004998132586479187,0.19068877398967743,35349.0,286,2025-12-17 13:24:59.288932,,,,,,,,, +0.1761743277311325,0.005162951070815325,0.1761743277311325,35399.0,286,2025-12-17 13:38:43.059581,,,,,,,,, +0.22528564929962158,0.006033760029822588,0.22528564929962158,35449.0,286,2025-12-17 13:49:25.828374,,,,,,,,, +,,,,286,2025-12-17 13:55:46.797140,0.18242765963077545,0.002836128929629922,0.18242765963077545,0.1781405359506607,0.0016534399474039674,0.1781405359506607,,, +,,,,286,2025-12-17 13:56:11.223084,,,,,,,0.1967431753873825,0.0031216940842568874,0.1967431753873825 +0.1713986098766327,0.001795358257368207,0.1713986098766327,35499.0,287,2025-12-17 14:07:13.490807,,,,,,,,, +0.22429664433002472,0.003829999826848507,0.22429664433002472,35549.0,287,2025-12-17 14:20:21.613220,,,,,,,,, +,,,,287,2025-12-17 14:35:22.155433,0.16012799739837646,0.002330467337742448,0.16012799739837646,0.19194453954696655,0.0028031340334564447,0.19194453954696655,,, +,,,,287,2025-12-17 14:35:46.172515,,,,,,,0.19873230159282684,0.0031826260965317488,0.19873230159282684 +0.20367878675460815,0.002648131689056754,0.20367878675460815,35599.0,288,2025-12-17 14:40:36.937022,,,,,,,,, +0.2237282395362854,0.0028633889742195606,0.2237282395362854,35649.0,288,2025-12-17 14:53:20.064598,,,,,,,,, +0.19779975712299347,0.0023645933251827955,0.19779975712299347,35699.0,288,2025-12-17 15:03:21.909385,,,,,,,,, +,,,,288,2025-12-17 15:10:36.093783,0.18665599822998047,0.002564403461292386,0.18665599822998047,0.18298223614692688,0.0037482294719666243,0.18298223614692688,,, +,,,,288,2025-12-17 15:11:00.875646,,,,,,,0.19629983603954315,0.0030719335190951824,0.19629983603954315 +0.1620388627052307,0.006648584268987179,0.1620388627052307,35749.0,289,2025-12-17 15:22:49.934776,,,,,,,,, +0.23760588467121124,0.0033967564813792706,0.23760588467121124,35799.0,289,2025-12-17 15:33:01.494301,,,,,,,,, +,,,,289,2025-12-17 15:47:57.166589,0.18343153595924377,0.002852681325748563,0.18343153595924377,0.17928771674633026,0.0023889360018074512,0.17928771674633026,,, +,,,,289,2025-12-17 15:48:20.971173,,,,,,,0.1942829042673111,0.0031304138246923685,0.1942829042673111 +0.2147243469953537,0.0023292461410164833,0.2147243469953537,35849.0,290,2025-12-17 15:53:29.208596,,,,,,,,, +0.17564254999160767,0.0012471943628042936,0.17564254999160767,35899.0,290,2025-12-17 16:09:15.949107,,,,,,,,, +0.20314127206802368,0.00508303614333272,0.20314127206802368,35949.0,290,2025-12-17 16:21:22.599269,,,,,,,,, +,,,,290,2025-12-17 16:27:05.852082,0.16923919320106506,0.0028303940780460835,0.16923919320106506,0.17788240313529968,0.0026354698929935694,0.17788240313529968,,, +,,,,290,2025-12-17 16:27:29.234106,,,,,,,0.19498877227306366,0.0027310862205922604,0.19498877227306366 +0.18682900071144104,0.005297146737575531,0.18682900071144104,35999.0,291,2025-12-17 16:38:01.258094,,,,,,,,, +0.21522513031959534,0.004911354277282953,0.21522513031959534,36049.0,291,2025-12-17 16:51:40.965703,,,,,,,,, +,,,,291,2025-12-17 17:04:42.576529,0.1743609607219696,0.0026003436651080847,0.1743609607219696,0.17897015810012817,0.00254379166290164,0.17897015810012817,,, +,,,,291,2025-12-17 17:05:07.305276,,,,,,,0.19865772128105164,0.0026552763301879168,0.19865772128105164 +0.17356239259243011,0.001469526207074523,0.17356239259243011,36099.0,292,2025-12-17 17:11:12.942321,,,,,,,,, +0.22174930572509766,0.004706332925707102,0.22174930572509766,36149.0,292,2025-12-17 17:28:01.108575,,,,,,,,, +0.20728912949562073,0.0013768320204690099,0.20728912949562073,36199.0,292,2025-12-17 17:43:41.802533,,,,,,,,, +,,,,292,2025-12-17 17:51:27.509741,0.1928902268409729,0.0029654884710907936,0.1928902268409729,0.1814386397600174,0.0027054366655647755,0.1814386397600174,,, +,,,,292,2025-12-17 17:51:52.914778,,,,,,,0.19806376099586487,0.003226323053240776,0.19806376099586487 +0.20805345475673676,0.0021228620316833258,0.20805345475673676,36249.0,293,2025-12-17 18:09:05.592814,,,,,,,,, +0.23357592523097992,0.0025804743636399508,0.23357592523097992,36299.0,293,2025-12-17 18:19:45.527649,,,,,,,,, +,,,,293,2025-12-17 18:30:15.249850,0.18621738255023956,0.002663141116499901,0.18621738255023956,0.18487092852592468,0.0031537015456706285,0.18487092852592468,,, +,,,,293,2025-12-17 18:30:39.937372,,,,,,,0.19924615323543549,0.003143022768199444,0.19924615323543549 +0.18185368180274963,0.002255125902593136,0.18185368180274963,36349.0,294,2025-12-17 18:38:35.458648,,,,,,,,, +0.18784427642822266,0.0014675428392365575,0.18784427642822266,36399.0,294,2025-12-17 18:48:43.482723,,,,,,,,, +0.16193780303001404,0.0010069446871057153,0.16193780303001404,36449.0,294,2025-12-17 18:59:20.213824,,,,,,,,, +,,,,294,2025-12-17 19:04:23.560347,0.18073086440563202,0.0030385239515453577,0.18073086440563202,0.1700720191001892,0.0024882375728338957,0.1700720191001892,,, +,,,,294,2025-12-17 19:05:09.138473,,,,,,,0.19555239379405975,0.002802817150950432,0.19555239379405975 +0.1841164529323578,0.0014150080969557166,0.1841164529323578,36499.0,295,2025-12-17 19:18:31.511227,,,,,,,,, +0.1802986115217209,0.005051462445408106,0.1802986115217209,36549.0,295,2025-12-17 19:35:07.651450,,,,,,,,, +,,,,295,2025-12-17 19:46:29.819620,0.1848856508731842,0.0028459273744374514,0.1848856508731842,0.17539693415164948,0.002648124471306801,0.17539693415164948,,, +,,,,295,2025-12-17 19:46:53.994141,,,,,,,0.19685092568397522,0.00318768247961998,0.19685092568397522 +0.19075939059257507,0.0030984808690845966,0.19075939059257507,36599.0,296,2025-12-17 19:58:06.465441,,,,,,,,, +0.20315372943878174,0.004348331596702337,0.20315372943878174,36649.0,296,2025-12-17 20:08:42.831354,,,,,,,,, +0.28807345032691956,0.004922538064420223,0.28807345032691956,36699.0,296,2025-12-17 20:20:06.102731,,,,,,,,, +,,,,296,2025-12-17 20:24:48.284063,0.17982029914855957,0.002245777752250433,0.17982029914855957,0.17800156772136688,0.0019393028924241662,0.17800156772136688,,, +,,,,296,2025-12-17 20:25:12.102581,,,,,,,0.19359882175922394,0.0029101031832396984,0.19359882175922394 +0.2032589316368103,0.00281696068122983,0.2032589316368103,36749.0,297,2025-12-17 20:39:26.791061,,,,,,,,, +0.21093781292438507,0.004346213769167662,0.21093781292438507,36799.0,297,2025-12-17 20:50:58.115846,,,,,,,,, +,,,,297,2025-12-17 21:03:50.253423,0.18996454775333405,0.003137686988338828,0.18996454775333405,0.1755760908126831,0.002408736851066351,0.1755760908126831,,, +,,,,297,2025-12-17 21:04:13.986625,,,,,,,0.19884474575519562,0.002881229156628251,0.19884474575519562 +0.21253341436386108,0.004194221924990416,0.21253341436386108,36849.0,298,2025-12-17 21:12:56.959870,,,,,,,,, +0.17210225760936737,0.0018689132994040847,0.17210225760936737,36899.0,298,2025-12-17 21:29:59.199303,,,,,,,,, +0.22049571573734283,0.0027918368577957153,0.22049571573734283,36949.0,298,2025-12-17 21:42:13.960718,,,,,,,,, +,,,,298,2025-12-17 21:47:02.448273,0.1824864149093628,0.003038965631276369,0.1824864149093628,0.18837711215019226,0.0024412923958152533,0.18837711215019226,,, +,,,,298,2025-12-17 21:47:26.310771,,,,,,,0.1969486027956009,0.002725606318563223,0.1969486027956009 +0.1532931923866272,0.0011586385080590844,0.1532931923866272,36999.0,299,2025-12-17 22:02:17.172834,,,,,,,,, +0.22986207902431488,0.00956023670732975,0.22986207902431488,37049.0,299,2025-12-17 22:15:20.865882,,,,,,,,, +,,,,299,2025-12-17 22:27:19.603201,0.18225140869617462,0.0028967135585844517,0.18225140869617462,0.17338544130325317,0.0029873677995055914,0.17338544130325317,,, +,,,,299,2025-12-17 22:27:44.141115,,,,,,,0.1947980374097824,0.0030728222336620092,0.1947980374097824 +0.19446338713169098,0.002393220318481326,0.19446338713169098,37099.0,300,2025-12-17 22:42:04.317410,,,,,,,,, +0.184031143784523,0.0015905434265732765,0.184031143784523,37149.0,300,2025-12-17 22:51:56.066609,,,,,,,,, +0.18347297608852386,0.002250293269753456,0.18347297608852386,37199.0,300,2025-12-17 23:05:42.981281,,,,,,,,, +,,,,300,2025-12-17 23:09:50.721928,0.17372164130210876,0.0019481725757941604,0.17372164130210876,0.17697125673294067,0.00249460618942976,0.17697125673294067,,, +,,,,300,2025-12-17 23:10:14.593633,,,,,,,0.19411447644233704,0.002939257537946105,0.19411447644233704 +0.182914599776268,0.0009741458925418556,0.182914599776268,37249.0,301,2025-12-17 23:22:37.874757,,,,,,,,, +0.21262986958026886,0.00256401882506907,0.21262986958026886,37299.0,301,2025-12-17 23:35:57.321618,,,,,,,,, +,,,,301,2025-12-17 23:46:19.834719,0.17736954987049103,0.0030306975822895765,0.17736954987049103,0.1863166093826294,0.002420314121991396,0.1863166093826294,,, +,,,,301,2025-12-17 23:46:43.931158,,,,,,,0.1941252201795578,0.003102425020188093,0.1941252201795578 +0.22488486766815186,0.002035267651081085,0.22488486766815186,37349.0,302,2025-12-17 23:56:09.660735,,,,,,,,, +0.2358729988336563,0.0038093766197562218,0.2358729988336563,37399.0,302,2025-12-18 00:06:48.535724,,,,,,,,, +,,,,302,2025-12-18 00:22:02.623835,0.18290169537067413,0.003877501469105482,0.18290169537067413,0.17657914757728577,0.002563844434916973,0.17657914757728577,,, +,,,,302,2025-12-18 00:22:27.748089,,,,,,,0.19547127187252045,0.002953862538561225,0.19547127187252045 +0.20837366580963135,0.005190796684473753,0.20837366580963135,37449.0,303,2025-12-18 00:29:57.583688,,,,,,,,, +0.19134950637817383,0.0012630264973267913,0.19134950637817383,37499.0,303,2025-12-18 00:40:05.786544,,,,,,,,, +0.18441659212112427,0.0028377845883369446,0.18441659212112427,37549.0,303,2025-12-18 00:52:12.586405,,,,,,,,, +,,,,303,2025-12-18 01:00:07.072663,0.18174448609352112,0.0033481563441455364,0.18174448609352112,0.17467130720615387,0.0023615104146301746,0.17467130720615387,,, +,,,,303,2025-12-18 01:00:31.390246,,,,,,,0.19815699756145477,0.0027375526260584593,0.19815699756145477 +0.18958479166030884,0.0023427943233400583,0.18958479166030884,37599.0,304,2025-12-18 01:12:05.926435,,,,,,,,, +0.21309299767017365,0.0041326116770505905,0.21309299767017365,37649.0,304,2025-12-18 01:23:24.305345,,,,,,,,, +,,,,304,2025-12-18 01:36:57.772953,0.17343847453594208,0.0024085587356239557,0.17343847453594208,0.18577489256858826,0.0032034795731306076,0.18577489256858826,,, +,,,,304,2025-12-18 01:37:22.326085,,,,,,,0.19561628997325897,0.0032590541522949934,0.19561628997325897 +0.18822786211967468,0.001705064787529409,0.18822786211967468,37699.0,305,2025-12-18 01:43:16.307334,,,,,,,,, +0.1839851289987564,0.0009338644449599087,0.1839851289987564,37749.0,305,2025-12-18 01:51:31.745080,,,,,,,,, +0.17194166779518127,0.001734840334393084,0.17194166779518127,37799.0,305,2025-12-18 02:09:39.947220,,,,,,,,, +,,,,305,2025-12-18 02:17:32.581838,0.18397285044193268,0.004496854729950428,0.18397285044193268,0.17991331219673157,0.002621322637423873,0.17991331219673157,,, +,,,,305,2025-12-18 02:17:56.499684,,,,,,,0.19706104695796967,0.0030260919593274593,0.19706104695796967 +0.21414990723133087,0.0026686997152864933,0.21414990723133087,37849.0,306,2025-12-18 02:30:27.361222,,,,,,,,, +0.19570845365524292,0.0017692528199404478,0.19570845365524292,37899.0,306,2025-12-18 02:44:51.811241,,,,,,,,, +,,,,306,2025-12-18 02:56:56.372770,0.18441125750541687,0.0024261779617518187,0.18441125750541687,0.17745493352413177,0.0024183292407542467,0.17745493352413177,,, +,,,,306,2025-12-18 02:57:21.538102,,,,,,,0.1987733244895935,0.0030451875645667315,0.1987733244895935 +0.16310657560825348,0.0035659947898238897,0.16310657560825348,37949.0,307,2025-12-18 03:01:32.578706,,,,,,,,, +0.17632657289505005,0.0010982691310346127,0.17632657289505005,37999.0,307,2025-12-18 03:16:31.634182,,,,,,,,, +0.1614169180393219,0.0036806236021220684,0.1614169180393219,38049.0,307,2025-12-18 03:29:40.929017,,,,,,,,, +,,,,307,2025-12-18 03:39:34.485962,0.17549605667591095,0.0029848015401512384,0.17549605667591095,0.17797605693340302,0.0024242717772722244,0.17797605693340302,,, +,,,,307,2025-12-18 03:39:59.377406,,,,,,,0.19625559449195862,0.0029788841493427753,0.19625559449195862 +0.2004111260175705,0.0015819529071450233,0.2004111260175705,38099.0,308,2025-12-18 03:50:19.189581,,,,,,,,, +0.1599348932504654,0.001278178533539176,0.1599348932504654,38149.0,308,2025-12-18 03:59:53.334827,,,,,,,,, +,,,,308,2025-12-18 04:15:42.177553,0.1929323971271515,0.0023959351237863302,0.1929323971271515,0.18436072766780853,0.0027598654851317406,0.18436072766780853,,, +,,,,308,2025-12-18 04:16:07.252565,,,,,,,0.19580480456352234,0.0028798070270568132,0.19580480456352234 +0.20839823782444,0.0017711512045934796,0.20839823782444,38199.0,309,2025-12-18 04:19:49.675400,,,,,,,,, +0.22588644921779633,0.0020249048247933388,0.22588644921779633,38249.0,309,2025-12-18 04:33:32.818373,,,,,,,,, +0.2363964319229126,0.0032642846927046776,0.2363964319229126,38299.0,309,2025-12-18 04:48:16.948615,,,,,,,,, +,,,,309,2025-12-18 04:54:59.181778,0.1860031932592392,0.002608959097415209,0.1860031932592392,0.18130889534950256,0.002833174541592598,0.18130889534950256,,, +,,,,309,2025-12-18 04:55:23.453103,,,,,,,0.19581033289432526,0.003426274284720421,0.19581033289432526 +0.17092517018318176,0.001044456847012043,0.17092517018318176,38349.0,310,2025-12-18 05:05:33.761769,,,,,,,,, +0.20168936252593994,0.0018790846224874258,0.20168936252593994,38399.0,310,2025-12-18 05:18:38.470136,,,,,,,,, +,,,,310,2025-12-18 05:30:47.895638,0.186677485704422,0.0028892189729958773,0.186677485704422,0.1768779456615448,0.0022392135579138994,0.1768779456615448,,, +,,,,310,2025-12-18 05:31:12.726468,,,,,,,0.19440361857414246,0.0028226561844348907,0.19440361857414246 +0.18738533556461334,0.007990417070686817,0.18738533556461334,38449.0,311,2025-12-18 05:35:52.623514,,,,,,,,, +0.21868625283241272,0.003124068956822157,0.21868625283241272,38499.0,311,2025-12-18 05:48:20.378205,,,,,,,,, +0.2365761697292328,0.003640596754848957,0.2365761697292328,38549.0,311,2025-12-18 05:56:38.048028,,,,,,,,, +,,,,311,2025-12-18 06:03:29.662191,0.17982910573482513,0.0019631460309028625,0.17982910573482513,0.18333543837070465,0.0030988880898803473,0.18333543837070465,,, +,,,,311,2025-12-18 06:03:53.770707,,,,,,,0.19213013350963593,0.002829383360221982,0.19213013350963593 +0.18557535111904144,0.004186674952507019,0.18557535111904144,38599.0,312,2025-12-18 06:15:57.837888,,,,,,,,, +0.17212362587451935,0.001110594253987074,0.17212362587451935,38649.0,312,2025-12-18 06:30:22.287676,,,,,,,,, +,,,,312,2025-12-18 06:43:11.131343,0.18025518953800201,0.0018081150483340025,0.18025518953800201,0.1767527312040329,0.002630329690873623,0.1767527312040329,,, +,,,,312,2025-12-18 06:43:35.943552,,,,,,,0.19421595335006714,0.0030011888593435287,0.19421595335006714 +0.14090505242347717,0.000825529161375016,0.14090505242347717,38699.0,313,2025-12-18 06:50:25.390321,,,,,,,,, +0.18135006725788116,0.0022085506934672594,0.18135006725788116,38749.0,313,2025-12-18 07:02:25.728077,,,,,,,,, +0.19928371906280518,0.0013971396256238222,0.19928371906280518,38799.0,313,2025-12-18 07:17:53.833522,,,,,,,,, +,,,,313,2025-12-18 07:24:20.370936,0.17466920614242554,0.00397008191794157,0.17466920614242554,0.17403990030288696,0.0020821786019951105,0.17403990030288696,,, +,,,,313,2025-12-18 07:24:44.794682,,,,,,,0.19571834802627563,0.0029935145284980536,0.19571834802627563 +0.20605291426181793,0.004720855969935656,0.20605291426181793,38849.0,314,2025-12-18 07:37:31.768083,,,,,,,,, +0.18560776114463806,0.0047049373388290405,0.18560776114463806,38899.0,314,2025-12-18 07:49:59.171406,,,,,,,,, +,,,,314,2025-12-18 08:03:37.520159,0.18141305446624756,0.002416597679257393,0.18141305446624756,0.18318989872932434,0.0020515206269919872,0.18318989872932434,,, +,,,,314,2025-12-18 08:04:01.690171,,,,,,,0.19592247903347015,0.0028697969391942024,0.19592247903347015 +0.174678236246109,0.0017831524601206183,0.174678236246109,38949.0,315,2025-12-18 08:08:45.533054,,,,,,,,, +0.18730443716049194,0.006899985484778881,0.18730443716049194,38999.0,315,2025-12-18 08:23:32.041541,,,,,,,,, +0.2113027274608612,0.006106391549110413,0.2113027274608612,39049.0,315,2025-12-18 08:36:23.057090,,,,,,,,, +,,,,315,2025-12-18 08:44:12.158628,0.17737950384616852,0.0030537156853824854,0.17737950384616852,0.18596240878105164,0.002707003615796566,0.18596240878105164,,, +,,,,315,2025-12-18 08:44:36.227210,,,,,,,0.19408559799194336,0.002935798605903983,0.19408559799194336 +0.23948663473129272,0.0025361673906445503,0.23948663473129272,39099.0,316,2025-12-18 08:56:50.367609,,,,,,,,, +0.2003689557313919,0.008366960100829601,0.2003689557313919,39149.0,316,2025-12-18 09:05:18.752954,,,,,,,,, +,,,,316,2025-12-18 09:19:56.289749,0.18243324756622314,0.00231556105427444,0.18243324756622314,0.1875740885734558,0.0021977988071739674,0.1875740885734558,,, +,,,,316,2025-12-18 09:20:21.606396,,,,,,,0.1990194171667099,0.0031759219709783792,0.1990194171667099 +0.18611517548561096,0.005217325873672962,0.18611517548561096,39199.0,317,2025-12-18 09:26:14.731551,,,,,,,,, +0.15701068937778473,0.0009074340341612697,0.15701068937778473,39249.0,317,2025-12-18 09:40:03.265842,,,,,,,,, +0.19729113578796387,0.005283483769744635,0.19729113578796387,39299.0,317,2025-12-18 09:59:03.298688,,,,,,,,, +,,,,317,2025-12-18 10:04:24.807265,0.17511440813541412,0.0024950362276285887,0.17511440813541412,0.17110250890254974,0.0033514914102852345,0.17110250890254974,,, +,,,,317,2025-12-18 10:05:12.452786,,,,,,,0.19516822695732117,0.0030038924887776375,0.19516822695732117 +0.16870637238025665,0.0013481976930052042,0.16870637238025665,39349.0,318,2025-12-18 10:20:32.530118,,,,,,,,, +0.25059017539024353,0.0023896561469882727,0.25059017539024353,39399.0,318,2025-12-18 10:32:16.113282,,,,,,,,, +,,,,318,2025-12-18 10:42:57.472191,0.17481006681919098,0.001979510299861431,0.17481006681919098,0.17789126932621002,0.002541458699852228,0.17789126932621002,,, +,,,,318,2025-12-18 10:43:21.729651,,,,,,,0.19543452560901642,0.0029914099723100662,0.19543452560901642 +0.1634989082813263,0.0030151938553899527,0.1634989082813263,39449.0,319,2025-12-18 10:52:07.734344,,,,,,,,, +0.21041148900985718,0.0019093843875452876,0.21041148900985718,39499.0,319,2025-12-18 11:04:01.029987,,,,,,,,, +0.20472106337547302,0.0017077079974114895,0.20472106337547302,39549.0,319,2025-12-18 11:16:26.614893,,,,,,,,, +,,,,319,2025-12-18 11:21:26.227179,0.17578238248825073,0.002984919585287571,0.17578238248825073,0.18494173884391785,0.002798818051815033,0.18494173884391785,,, +,,,,319,2025-12-18 11:21:51.392918,,,,,,,0.19589555263519287,0.002925304463133216,0.19589555263519287 +0.2144615650177002,0.0028618969954550266,0.2144615650177002,39599.0,320,2025-12-18 11:34:45.464201,,,,,,,,, +0.1992233246564865,0.0016374657861888409,0.1992233246564865,39649.0,320,2025-12-18 11:50:46.269871,,,,,,,,, +,,,,320,2025-12-18 12:00:13.248534,0.19001175463199615,0.002462134463712573,0.19001175463199615,0.17629726231098175,0.002196073066443205,0.17629726231098175,,, +,,,,320,2025-12-18 12:00:37.542670,,,,,,,0.1944391429424286,0.002750263549387455,0.1944391429424286 +0.1701757162809372,0.0011884849518537521,0.1701757162809372,39699.0,321,2025-12-18 12:06:37.732702,,,,,,,,, +0.18713952600955963,0.0019422051263973117,0.18713952600955963,39749.0,321,2025-12-18 12:18:05.474766,,,,,,,,, +0.2064806967973709,0.001708492636680603,0.2064806967973709,39799.0,321,2025-12-18 12:28:44.272691,,,,,,,,, +,,,,321,2025-12-18 12:33:28.202133,0.17638371884822845,0.0021646416280418634,0.17638371884822845,0.1839720457792282,0.002940575359389186,0.1839720457792282,,, +,,,,321,2025-12-18 12:33:54.375636,,,,,,,0.19675534963607788,0.003171149641275406,0.19675534963607788 +0.19413569569587708,0.0012798677198588848,0.19413569569587708,39849.0,322,2025-12-18 12:50:56.571797,,,,,,,,, +0.15306596457958221,0.0011883355909958482,0.15306596457958221,39899.0,322,2025-12-18 13:03:52.595521,,,,,,,,, +,,,,322,2025-12-18 13:15:07.131350,0.17302660644054413,0.002749087754637003,0.17302660644054413,0.17609232664108276,0.002592393895611167,0.17609232664108276,,, +,,,,322,2025-12-18 13:15:31.741569,,,,,,,0.19455388188362122,0.002842786256223917,0.19455388188362122 +0.2301112413406372,0.005874329712241888,0.2301112413406372,39949.0,323,2025-12-18 13:23:32.532750,,,,,,,,, +0.1635185331106186,0.0043691168539226055,0.1635185331106186,39999.0,323,2025-12-18 13:33:39.256537,,,,,,,,, +0.21078847348690033,0.0026073495391756296,0.21078847348690033,40049.0,323,2025-12-18 13:45:36.232695,,,,,,,,, +,,,,323,2025-12-18 13:50:02.723210,0.17930857837200165,0.0035674930550158024,0.17930857837200165,0.17814700305461884,0.002707277424633503,0.17814700305461884,,, +,,,,323,2025-12-18 13:50:27.171030,,,,,,,0.19482940435409546,0.002966072177514434,0.19482940435409546 +0.19115252792835236,0.0017317520687356591,0.19115252792835236,40099.0,324,2025-12-18 14:01:47.258693,,,,,,,,, +0.1511041671037674,0.003961846232414246,0.1511041671037674,40149.0,324,2025-12-18 14:16:00.332350,,,,,,,,, +,,,,324,2025-12-18 14:27:09.110130,0.1806202381849289,0.0028449438977986574,0.1806202381849289,0.17929379642009735,0.0021106500644236803,0.17929379642009735,,, +,,,,324,2025-12-18 14:27:34.587060,,,,,,,0.1949233114719391,0.0030019234400242567,0.1949233114719391 +0.19914166629314423,0.005113033577799797,0.19914166629314423,40199.0,325,2025-12-18 14:35:04.523920,,,,,,,,, +0.14676980674266815,0.0010466431267559528,0.14676980674266815,40249.0,325,2025-12-18 14:45:19.879912,,,,,,,,, +0.20381130278110504,0.004115038085728884,0.20381130278110504,40299.0,325,2025-12-18 14:57:04.498235,,,,,,,,, +,,,,325,2025-12-18 15:01:08.457378,0.17321757972240448,0.0031391202937811613,0.17321757972240448,0.1735127866268158,0.0020058914087712765,0.1735127866268158,,, +,,,,325,2025-12-18 15:01:32.571457,,,,,,,0.19644002616405487,0.0034628366120159626,0.19644002616405487 +0.15725864470005035,0.00265692756511271,0.15725864470005035,40349.0,326,2025-12-18 15:14:40.296621,,,,,,,,, +0.1851225346326828,0.0010800074087455869,0.1851225346326828,40399.0,326,2025-12-18 15:27:20.132612,,,,,,,,, +,,,,326,2025-12-18 15:35:19.852191,0.17455080151557922,0.0025724931620061398,0.17455080151557922,0.17855267226696014,0.003291358007118106,0.17855267226696014,,, +,,,,326,2025-12-18 15:35:43.873840,,,,,,,0.19473886489868164,0.002894595032557845,0.19473886489868164 +0.17003604769706726,0.0016836343565955758,0.17003604769706726,40449.0,327,2025-12-18 15:44:32.244420,,,,,,,,, +0.2301541268825531,0.002312125638127327,0.2301541268825531,40499.0,327,2025-12-18 15:52:48.061496,,,,,,,,, +,,,,327,2025-12-18 16:10:13.013144,0.18270128965377808,0.0029150487389415503,0.18270128965377808,0.17789238691329956,0.002020481741055846,0.17789238691329956,,, +,,,,327,2025-12-18 16:10:38.756498,,,,,,,0.1940612643957138,0.00283252471126616,0.1940612643957138 +0.20453684031963348,0.0027112187817692757,0.20453684031963348,40549.0,328,2025-12-18 16:14:23.941795,,,,,,,,, +0.1748676300048828,0.0029278784058988094,0.1748676300048828,40599.0,328,2025-12-18 16:25:17.499373,,,,,,,,, +0.18326467275619507,0.004852099809795618,0.18326467275619507,40649.0,328,2025-12-18 16:38:45.794856,,,,,,,,, +,,,,328,2025-12-18 16:46:27.280642,0.18736249208450317,0.0030246691312640905,0.18736249208450317,0.1861990988254547,0.002888748422265053,0.1861990988254547,,, +,,,,328,2025-12-18 16:46:52.533740,,,,,,,0.19074462354183197,0.0031001903116703033,0.19074462354183197 +0.20639023184776306,0.005100106820464134,0.20639023184776306,40699.0,329,2025-12-18 16:59:09.857916,,,,,,,,, +0.1912497878074646,0.0019827138166874647,0.1912497878074646,40749.0,329,2025-12-18 17:09:45.759884,,,,,,,,, +,,,,329,2025-12-18 17:22:40.573432,0.17840462923049927,0.0028951268177479506,0.17840462923049927,0.1829763948917389,0.0022093721199780703,0.1829763948917389,,, +,,,,329,2025-12-18 17:23:04.594409,,,,,,,0.19599395990371704,0.0031697291415184736,0.19599395990371704 +0.21093139052391052,0.008723374456167221,0.21093139052391052,40799.0,330,2025-12-18 17:27:00.185450,,,,,,,,, +0.1533731371164322,0.0010209032334387302,0.1533731371164322,40849.0,330,2025-12-18 17:40:33.443056,,,,,,,,, +0.2005375325679779,0.001744328998029232,0.2005375325679779,40899.0,330,2025-12-18 17:52:42.518686,,,,,,,,, +,,,,330,2025-12-18 18:02:47.918674,0.1736074686050415,0.0028950173873454332,0.1736074686050415,0.17256104946136475,0.0017255485290661454,0.17256104946136475,,, +,,,,330,2025-12-18 18:03:11.780690,,,,,,,0.19547094404697418,0.002996263327077031,0.19547094404697418 +0.16243186593055725,0.002949012676253915,0.16243186593055725,40949.0,331,2025-12-18 18:13:09.671737,,,,,,,,, +0.18662923574447632,0.0021973317489027977,0.18662923574447632,40999.0,331,2025-12-18 18:24:24.282104,,,,,,,,, +,,,,331,2025-12-18 18:37:39.573805,0.17542533576488495,0.0024888403713703156,0.17542533576488495,0.18507540225982666,0.0023641204461455345,0.18507540225982666,,, +,,,,331,2025-12-18 18:38:03.727420,,,,,,,0.19023220241069794,0.003119408618658781,0.19023220241069794 +0.23441451787948608,0.0021940816659480333,0.23441451787948608,41049.0,332,2025-12-18 18:42:34.239325,,,,,,,,, +0.2027035355567932,0.005364261567592621,0.2027035355567932,41099.0,332,2025-12-18 18:51:19.353630,,,,,,,,, +0.2025831639766693,0.001039035152643919,0.2025831639766693,41149.0,332,2025-12-18 19:05:01.895077,,,,,,,,, diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/tf/events.out.tfevents.1765489805.node-0.2034.0 b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/tf/events.out.tfevents.1765489805.node-0.2034.0 new file mode 100644 index 0000000000000000000000000000000000000000..230eedd9730453438a94ff99398847f03a1b4f11 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_2d_with_body_mask2/testtube/version_3/tf/events.out.tfevents.1765489805.node-0.2034.0 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:42b195cb20ea174baa797d8a0699b5f79eefc2f94f946edcbd47b2ad674c81e5 +size 324697 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/epoch=000013.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/epoch=000013.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..82b4e95d7f8aeb0fa8990b1e4610ac48cd2340e7 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/epoch=000013.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e04ff965787c429fda05060d1a6397ca6164accd3d16ecb4f6bc84bf93440f4a +size 3016767037 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/epoch=000061.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/epoch=000061.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..0d33a0a53ddf5e4fa4f457ace36576d4852cd917 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/epoch=000061.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:594107fa9e26c86a543be3bc92c76e0dd8b38adeba36aefd35af6511b646de91 +size 3016767037 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/epoch=000095.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/epoch=000095.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..9009bcf0f928bf3acd39601a9de5d99bb6d40e5b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/epoch=000095.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3f93576617806b661f921b26df766d4df234381c9f1edfa555be5b63b04a23a +size 3016767037 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/last.ckpt b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/last.ckpt new file mode 100644 index 0000000000000000000000000000000000000000..4489b5b17e6a88effa6382bbcdea54059dd8f355 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/checkpoints/last.ckpt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b0b9d8f1973e3c20fdcddfc4d71a4f4f737e956a9c11a49e6b138aa36096640d +size 3016767037 diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/configs/2025-12-18T19-35-00-lightning.yaml b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/configs/2025-12-18T19-35-00-lightning.yaml new file mode 100644 index 0000000000000000000000000000000000000000..af576c4f4222ea130206951d1e6274cd32183a05 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/configs/2025-12-18T19-35-00-lightning.yaml @@ -0,0 +1,12 @@ +lightning: + callbacks: + image_logger: + target: main.ImageLogger + params: + batch_frequency: 100000000 + max_images: 16 + increase_log_steps: false + trainer: + benchmark: true + accelerator: ddp + gpus: 0,1,2,3, diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/configs/2025-12-18T19-35-00-project.yaml b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/configs/2025-12-18T19-35-00-project.yaml new file mode 100644 index 0000000000000000000000000000000000000000..7b773de981b8dfeef02ac79b87daa998ac342b0f --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/configs/2025-12-18T19-35-00-project.yaml @@ -0,0 +1,61 @@ +model: + base_learning_rate: 5.0e-06 + target: ldm.models.diffusion.ddpm_pseudo3D.LatentDiffusion + params: + linear_start: 0.00085 + linear_end: 0.012 + num_timesteps_cond: 1 + log_every_t: 200 + timesteps: 1000 + first_stage_key: volume_data + cond_stage_key: volume_seg_and_text + conditioning_key: crossattn + text_enc: custom + image_size: 64 + channels: 16 + monitor: val/loss_simple_ema + scale_factor: 1.52 + use_ema: false + ckpt_path: /sd/qichen/full_ct_gen/GEM-3D-ct-text4-newdata-newvae-all3-fix/logs/full_ct_2d_with_body_mask2/checkpoints/epoch=000317.ckpt + load_only_unet: true + fix_t: true + unet_config: + target: ldm.modules.diffusionmodules.openaimodel_pseudo3D.UNetModel + params: + image_size: 64 + in_channels: 32 + out_channels: 16 + model_channels: 224 + attention_resolutions: + - 8 + - 4 + - 2 + num_res_blocks: 2 + channel_mult: + - 1 + - 2 + - 3 + - 4 + num_head_channels: 32 + use_spatial_transformer: true + context_dim: 768 + transformer_depth: 1 + use_checkpoint: true + legacy: false +data: + target: main.DataModuleFromConfig + params: + batch_size: 4 + num_workers: 8 + wrap: false + train: + target: ldm.data.ct_clip_data_train.CTReportDataset + params: + data_folder: /sd/shuhan/CT-RATE/dataset/train_fixed + csv_file: /sd/shuhan/CT-RATE/radiology_text_reports/train_reports.csv + validation: + target: ldm.data.ct_clip_data_inference.CTReportDatasetinfer + params: + data_folder: /sd/shuhan/CT-RATE/dataset/valid_fixed + csv_file: /sd/shuhan/CT-RATE/radiology_text_reports/valid_reports.csv + labels: /sd/shuhan/CT-RATE/multi_abnormality_labels/valid_predicted_labels.csv diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/meta.experiment b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/meta.experiment new file mode 100644 index 0000000000000000000000000000000000000000..6b33f5cfc1e0739f76433a25e66d45a89e105567 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/meta.experiment @@ -0,0 +1 @@ +{"name": "testtube", "version": 0, "tags_path": "logs/full_ct_3d_with_body_mask2/testtube/version_0/meta_tags.csv", "metrics_path": "logs/full_ct_3d_with_body_mask2/testtube/version_0/metrics.csv", "autosave": false, "description": null, "created_at": "2025-12-18 19:35:30.143376", "exp_hash": "testtube_v0"} \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/meta_tags.csv b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/meta_tags.csv new file mode 100644 index 0000000000000000000000000000000000000000..942a47139f86778b1f3e5ad923104dd038b1ce88 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/meta_tags.csv @@ -0,0 +1 @@ +key,value diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/metrics.csv b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/metrics.csv new file mode 100644 index 0000000000000000000000000000000000000000..ca2765ce73e757a967b20f2e7c24b41bdb91eb85 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/metrics.csv @@ -0,0 +1,746 @@ +train/loss_simple_step,train/loss_vlb_step,train/loss_step,global_step,epoch,created_at,val/loss_simple,val/loss_vlb,val/loss,val/loss_simple_ema,val/loss_vlb_ema,val/loss_ema,train/loss_simple_epoch,train/loss_vlb_epoch,train/loss_epoch +0.11295471340417862,0.0005624820478260517,0.11295471340417862,49.0,0,2025-12-18 19:54:15.657566,,,,,,,,, +0.2288961410522461,0.0017227407079190016,0.2288961410522461,99.0,0,2025-12-18 20:02:20.170736,,,,,,,,, +0.331623911857605,0.0018277105409651995,0.331623911857605,149.0,0,2025-12-18 20:11:37.199467,,,,,,,,, +0.11447180807590485,0.00039871621993370354,0.11447180807590485,199.0,0,2025-12-18 20:27:37.617247,,,,,,,,, +,,,,0,2025-12-18 20:41:27.833621,0.19654937088489532,0.0016020659822970629,0.19654937088489532,0.23785631358623505,0.009776003658771515,0.23785631358623505,,, +,,,,0,2025-12-18 20:41:48.522235,,,,,,,0.23315784335136414,0.0037488711532205343,0.23315784335136414 +0.3087160885334015,0.011789105832576752,0.3087160885334015,249.0,1,2025-12-18 20:43:17.304083,,,,,,,,, +0.16676174104213715,0.0005591522785834968,0.16676174104213715,299.0,1,2025-12-18 20:53:55.850467,,,,,,,,, +0.26891911029815674,0.0021485397592186928,0.26891911029815674,349.0,1,2025-12-18 21:09:14.803910,,,,,,,,, +0.2006152868270874,0.0007253707153722644,0.2006152868270874,399.0,1,2025-12-18 21:19:55.486388,,,,,,,,, +0.3445734977722168,0.0019794313702732325,0.3445734977722168,449.0,1,2025-12-18 21:33:35.871079,,,,,,,,, +,,,,1,2025-12-18 21:55:27.932510,0.17723675072193146,0.0012351013720035553,0.17723675072193146,0.20802649855613708,0.0024215709418058395,0.20802649855613708,,, +,,,,1,2025-12-18 21:55:50.443254,,,,,,,0.23223251104354858,0.002908039605244994,0.23223251104354858 +0.28849366307258606,0.0017287194496020675,0.28849366307258606,499.0,2,2025-12-18 21:57:27.565235,,,,,,,,, +0.12906204164028168,0.0005301121273078024,0.12906204164028168,549.0,2,2025-12-18 22:06:33.591818,,,,,,,,, +0.3300454616546631,0.0033610891550779343,0.3300454616546631,599.0,2,2025-12-18 22:19:00.432209,,,,,,,,, +0.2397991418838501,0.0017968052998185158,0.2397991418838501,649.0,2,2025-12-18 22:33:15.477739,,,,,,,,, +0.14896711707115173,0.0006664369138889015,0.14896711707115173,699.0,2,2025-12-18 22:48:07.404863,,,,,,,,, +,,,,2,2025-12-18 23:05:20.253744,0.1963701695203781,0.00112074741628021,0.1963701695203781,0.18875248730182648,0.0023080692626535892,0.18875248730182648,,, +,,,,2,2025-12-18 23:05:39.795175,,,,,,,0.22864161431789398,0.0037205289117991924,0.22864161431789398 +0.3798057436943054,0.031138328835368156,0.3798057436943054,749.0,3,2025-12-18 23:09:29.620822,,,,,,,,, +0.19400109350681305,0.000767063582316041,0.19400109350681305,799.0,3,2025-12-18 23:20:42.650266,,,,,,,,, +0.14312335848808289,0.00071352394297719,0.14312335848808289,849.0,3,2025-12-18 23:32:47.858079,,,,,,,,, +0.22875329852104187,0.0009639450581744313,0.22875329852104187,899.0,3,2025-12-18 23:47:46.782994,,,,,,,,, +0.4647781550884247,0.10246104001998901,0.4647781550884247,949.0,3,2025-12-19 00:01:36.319260,,,,,,,,, +,,,,3,2025-12-19 00:15:34.044420,0.2508760392665863,0.0023568759206682444,0.2508760392665863,0.20243221521377563,0.003754184115678072,0.20243221521377563,,, +,,,,3,2025-12-19 00:15:55.084647,,,,,,,0.2271413803100586,0.005432735197246075,0.2271413803100586 +0.2634313106536865,0.002866052556782961,0.2634313106536865,999.0,4,2025-12-19 00:21:10.122126,,,,,,,,, +0.41942137479782104,0.0019032289274036884,0.41942137479782104,1049.0,4,2025-12-19 00:31:24.634002,,,,,,,,, +0.33348309993743896,0.00146081845741719,0.33348309993743896,1099.0,4,2025-12-19 00:42:25.426022,,,,,,,,, +0.2059360146522522,0.0007775280391797423,0.2059360146522522,1149.0,4,2025-12-19 00:51:52.684788,,,,,,,,, +0.2581804394721985,0.0010640161344781518,0.2581804394721985,1199.0,4,2025-12-19 01:03:39.266319,,,,,,,,, +,,,,4,2025-12-19 01:18:23.429349,0.23815971612930298,0.0014813556335866451,0.23815971612930298,0.2397611141204834,0.0021460929419845343,0.2397611141204834,,, +,,,,4,2025-12-19 01:18:34.632349,,,,,,,0.23777051270008087,0.005844057537615299,0.23777051270008087 +0.22450731694698334,0.002664761384949088,0.22450731694698334,1249.0,5,2025-12-19 01:22:30.357997,,,,,,,,, +0.10651307553052902,0.0005135812098160386,0.10651307553052902,1299.0,5,2025-12-19 01:37:07.578777,,,,,,,,, +0.38231992721557617,0.0022244874853640795,0.38231992721557617,1349.0,5,2025-12-19 01:48:04.186197,,,,,,,,, +0.2888961136341095,0.0013128098798915744,0.2888961136341095,1399.0,5,2025-12-19 02:01:34.110915,,,,,,,,, +0.5767573118209839,0.01019372045993805,0.5767573118209839,1449.0,5,2025-12-19 02:10:50.844736,,,,,,,,, +,,,,5,2025-12-19 02:26:12.738687,0.1854245960712433,0.001980398315936327,0.1854245960712433,0.22972095012664795,0.0024753238540142775,0.22972095012664795,,, +,,,,5,2025-12-19 02:26:24.459583,,,,,,,0.2245520055294037,0.004195182118564844,0.2245520055294037 +0.5032626986503601,0.005847143940627575,0.5032626986503601,1499.0,6,2025-12-19 02:28:55.074367,,,,,,,,, +0.17652760446071625,0.004506699275225401,0.17652760446071625,1549.0,6,2025-12-19 02:40:07.665409,,,,,,,,, +0.03640907630324364,0.00013719465641770512,0.03640907630324364,1599.0,6,2025-12-19 02:53:10.647576,,,,,,,,, +0.33194106817245483,0.001469576614908874,0.33194106817245483,1649.0,6,2025-12-19 03:04:06.577016,,,,,,,,, +0.30092597007751465,0.00551483454182744,0.30092597007751465,1699.0,6,2025-12-19 03:13:48.275780,,,,,,,,, +,,,,6,2025-12-19 03:29:10.588386,0.22311392426490784,0.0016549652209505439,0.22311392426490784,0.1833496242761612,0.0021031356882303953,0.1833496242761612,,, +,,,,6,2025-12-19 03:29:32.054595,,,,,,,0.22413919866085052,0.003383755451068282,0.22413919866085052 +0.20501674711704254,0.000772338651586324,0.20501674711704254,1749.0,7,2025-12-19 03:36:31.218247,,,,,,,,, +0.19825448095798492,0.0009968782542273402,0.19825448095798492,1799.0,7,2025-12-19 03:45:49.041368,,,,,,,,, +0.13318711519241333,0.0004709338827524334,0.13318711519241333,1849.0,7,2025-12-19 03:59:07.107340,,,,,,,,, +0.2974013090133667,0.0020788179244846106,0.2974013090133667,1899.0,7,2025-12-19 04:08:14.342680,,,,,,,,, +0.2104284018278122,0.0007884346996434033,0.2104284018278122,1949.0,7,2025-12-19 04:19:47.870406,,,,,,,,, +,,,,7,2025-12-19 04:36:45.399250,0.21880382299423218,0.0013893508585169911,0.21880382299423218,0.20023208856582642,0.00864382553845644,0.20023208856582642,,, +,,,,7,2025-12-19 04:37:08.145259,,,,,,,0.2376498281955719,0.003989483695477247,0.2376498281955719 +0.32907694578170776,0.001618162845261395,0.32907694578170776,1999.0,8,2025-12-19 04:42:57.315882,,,,,,,,, +0.21343287825584412,0.0011187561322003603,0.21343287825584412,2049.0,8,2025-12-19 04:55:03.294232,,,,,,,,, +0.1432022750377655,0.0007543384563177824,0.1432022750377655,2099.0,8,2025-12-19 05:04:35.226166,,,,,,,,, +0.2776816189289093,0.0013705170713365078,0.2776816189289093,2149.0,8,2025-12-19 05:16:12.213551,,,,,,,,, +0.20891180634498596,0.006430622655898333,0.20891180634498596,2199.0,8,2025-12-19 05:27:47.549882,,,,,,,,, +,,,,8,2025-12-19 05:46:09.706498,0.20461906492710114,0.0023661181330680847,0.20461906492710114,0.22677725553512573,0.008014445193111897,0.22677725553512573,,, +,,,,8,2025-12-19 05:46:21.566812,,,,,,,0.21708929538726807,0.004568521399050951,0.21708929538726807 +0.3410705327987671,0.0050340979360044,0.3410705327987671,2249.0,9,2025-12-19 05:51:35.978681,,,,,,,,, +0.19590996205806732,0.0013436083681881428,0.19590996205806732,2299.0,9,2025-12-19 06:00:10.345400,,,,,,,,, +0.022066321223974228,9.647165279602632e-05,0.022066321223974228,2349.0,9,2025-12-19 06:16:14.527577,,,,,,,,, +0.23087546229362488,0.0010150668676942587,0.23087546229362488,2399.0,9,2025-12-19 06:23:14.249656,,,,,,,,, +0.14577531814575195,0.0005062671261839569,0.14577531814575195,2449.0,9,2025-12-19 06:33:07.174574,,,,,,,,, +,,,,9,2025-12-19 06:43:45.066723,0.1747010201215744,0.0013761352747678757,0.1747010201215744,0.19559940695762634,0.0017837804043665528,0.19559940695762634,,, +,,,,9,2025-12-19 06:44:06.580251,,,,,,,0.21955251693725586,0.002620029030367732,0.21955251693725586 +0.25017285346984863,0.005065167788416147,0.25017285346984863,2499.0,10,2025-12-19 06:50:26.368779,,,,,,,,, +0.26209932565689087,0.0010605285642668605,0.26209932565689087,2549.0,10,2025-12-19 07:04:54.001280,,,,,,,,, +0.1854596883058548,0.0006532547413371503,0.1854596883058548,2599.0,10,2025-12-19 07:18:09.145281,,,,,,,,, +0.266205370426178,0.0015646180836483836,0.266205370426178,2649.0,10,2025-12-19 07:28:16.897096,,,,,,,,, +0.14683184027671814,0.0004950165748596191,0.14683184027671814,2699.0,10,2025-12-19 07:37:44.768196,,,,,,,,, +,,,,10,2025-12-19 07:49:36.818077,0.2022724747657776,0.0014982193242758512,0.2022724747657776,0.2109186351299286,0.0018739711958914995,0.2109186351299286,,, +,,,,10,2025-12-19 07:49:48.706636,,,,,,,0.24352708458900452,0.003910996951162815,0.24352708458900452 +0.42209282517433167,0.0058427839539945126,0.42209282517433167,2749.0,11,2025-12-19 08:00:36.617768,,,,,,,,, +0.19531959295272827,0.0007596937357448041,0.19531959295272827,2799.0,11,2025-12-19 08:11:15.960398,,,,,,,,, +0.2634819746017456,0.0031775222159922123,0.2634819746017456,2849.0,11,2025-12-19 08:21:09.909620,,,,,,,,, +0.46045976877212524,0.004282113164663315,0.46045976877212524,2899.0,11,2025-12-19 08:32:47.257741,,,,,,,,, +0.2615104019641876,0.0018085287883877754,0.2615104019641876,2949.0,11,2025-12-19 08:43:22.112111,,,,,,,,, +,,,,11,2025-12-19 08:52:30.926344,0.270524799823761,0.004697940777987242,0.270524799823761,0.25093063712120056,0.004743491765111685,0.25093063712120056,,, +,,,,11,2025-12-19 08:52:42.177500,,,,,,,0.22462616860866547,0.0033263741061091423,0.22462616860866547 +0.12534476816654205,0.0005594590329565108,0.12534476816654205,2999.0,12,2025-12-19 08:59:50.139718,,,,,,,,, +0.0866248682141304,0.0003092258994001895,0.0866248682141304,3049.0,12,2025-12-19 09:10:26.592070,,,,,,,,, +0.28133466839790344,0.0010987393325194716,0.28133466839790344,3099.0,12,2025-12-19 09:24:31.572553,,,,,,,,, +0.12154944986104965,0.00048165986663661897,0.12154944986104965,3149.0,12,2025-12-19 09:36:28.953623,,,,,,,,, +0.4416932761669159,0.0032044127583503723,0.4416932761669159,3199.0,12,2025-12-19 09:48:05.448685,,,,,,,,, +,,,,12,2025-12-19 09:59:03.785730,0.18361487984657288,0.0010849997634068131,0.18361487984657288,0.23222924768924713,0.0042919740080833435,0.23222924768924713,,, +,,,,12,2025-12-19 09:59:15.511363,,,,,,,0.23434434831142426,0.0033401960972696543,0.23434434831142426 +0.2994135022163391,0.002951576840132475,0.2994135022163391,3249.0,13,2025-12-19 10:08:49.293339,,,,,,,,, +0.19922514259815216,0.0011123839067295194,0.19922514259815216,3299.0,13,2025-12-19 10:25:58.320335,,,,,,,,, +0.27357804775238037,0.0012375368969514966,0.27357804775238037,3349.0,13,2025-12-19 10:37:10.851657,,,,,,,,, +0.2295471578836441,0.0019295965321362019,0.2295471578836441,3399.0,13,2025-12-19 10:48:44.710723,,,,,,,,, +0.27930396795272827,0.0016719267005100846,0.27930396795272827,3449.0,13,2025-12-19 11:03:28.729506,,,,,,,,, +,,,,13,2025-12-19 11:11:55.774137,0.19584372639656067,0.001649295911192894,0.19584372639656067,0.16444607079029083,0.0009128681849688292,0.16444607079029083,,, +,,,,13,2025-12-19 11:12:19.550265,,,,,,,0.22202880680561066,0.002934363204985857,0.22202880680561066 +0.2316662222146988,0.0010306789772585034,0.2316662222146988,3499.0,14,2025-12-19 11:18:31.334007,,,,,,,,, +0.16667917370796204,0.0024761916138231754,0.16667917370796204,3549.0,14,2025-12-19 11:29:37.554565,,,,,,,,, +0.17544789612293243,0.000988606596365571,0.17544789612293243,3599.0,14,2025-12-19 11:39:15.748357,,,,,,,,, +0.2970344126224518,0.001342745148576796,0.2970344126224518,3649.0,14,2025-12-19 11:54:31.657118,,,,,,,,, +0.44336679577827454,0.0030542321037501097,0.44336679577827454,3699.0,14,2025-12-19 12:05:11.441300,,,,,,,,, +,,,,14,2025-12-19 12:13:06.914032,0.20606878399848938,0.0018387084128335118,0.20606878399848938,0.22380304336547852,0.001979984575882554,0.22380304336547852,,, +,,,,14,2025-12-19 12:13:19.121524,,,,,,,0.23582015931606293,0.003982486668974161,0.23582015931606293 +0.23256196081638336,0.001014266163110733,0.23256196081638336,3749.0,15,2025-12-19 12:22:09.526481,,,,,,,,, +0.11528579145669937,0.0003899935691151768,0.11528579145669937,3799.0,15,2025-12-19 12:31:27.505962,,,,,,,,, +0.3558900058269501,0.001808139611966908,0.3558900058269501,3849.0,15,2025-12-19 12:43:23.726133,,,,,,,,, +0.13118721544742584,0.00043413604726083577,0.13118721544742584,3899.0,15,2025-12-19 12:51:32.019149,,,,,,,,, +0.09165193885564804,0.00031643352122046053,0.09165193885564804,3949.0,15,2025-12-19 13:01:42.407521,,,,,,,,, +,,,,15,2025-12-19 13:08:41.822013,0.20447838306427002,0.0015026447363197803,0.20447838306427002,0.19977667927742004,0.0031905637588351965,0.19977667927742004,,, +,,,,15,2025-12-19 13:08:53.566874,,,,,,,0.23823238909244537,0.0038553106132894754,0.23823238909244537 +0.2837364673614502,0.001090075820684433,0.2837364673614502,3999.0,16,2025-12-19 13:15:43.527888,,,,,,,,, +0.35377243161201477,0.004922518506646156,0.35377243161201477,4049.0,16,2025-12-19 13:23:39.527927,,,,,,,,, +0.13396042585372925,0.0005088899051770568,0.13396042585372925,4099.0,16,2025-12-19 13:36:24.419608,,,,,,,,, +0.34798020124435425,0.002548038028180599,0.34798020124435425,4149.0,16,2025-12-19 13:48:43.025879,,,,,,,,, +0.16350682079792023,0.0006295942002907395,0.16350682079792023,4199.0,16,2025-12-19 13:59:03.385859,,,,,,,,, +,,,,16,2025-12-19 14:06:28.187684,0.22318722307682037,0.0034679763484746218,0.22318722307682037,0.23850709199905396,0.001723851659335196,0.23850709199905396,,, +,,,,16,2025-12-19 14:06:39.532304,,,,,,,0.2177266627550125,0.002799572655931115,0.2177266627550125 +0.08158741146326065,0.0002862226974684745,0.08158741146326065,4249.0,17,2025-12-19 14:12:31.891979,,,,,,,,, +0.32089099287986755,0.0016406434588134289,0.32089099287986755,4299.0,17,2025-12-19 14:23:12.884975,,,,,,,,, +0.12117702513933182,0.0005610077641904354,0.12117702513933182,4349.0,17,2025-12-19 14:37:12.639184,,,,,,,,, +0.14524221420288086,0.000485508528072387,0.14524221420288086,4399.0,17,2025-12-19 14:49:43.975780,,,,,,,,, +0.1948930025100708,0.0010098433122038841,0.1948930025100708,4449.0,17,2025-12-19 14:56:21.707158,,,,,,,,, +,,,,17,2025-12-19 15:03:39.844961,0.23459139466285706,0.003614019602537155,0.23459139466285706,0.20806705951690674,0.0017815155442804098,0.20806705951690674,,, +,,,,17,2025-12-19 15:03:51.636576,,,,,,,0.2297944575548172,0.0027668511029332876,0.2297944575548172 +0.15513551235198975,0.0006201682263053954,0.15513551235198975,4499.0,18,2025-12-19 15:13:05.879133,,,,,,,,, +0.34887734055519104,0.0016766502521932125,0.34887734055519104,4549.0,18,2025-12-19 15:26:03.495466,,,,,,,,, +0.17555831372737885,0.0007641340489499271,0.17555831372737885,4599.0,18,2025-12-19 15:36:46.688263,,,,,,,,, +0.2122005671262741,0.0007273319642990828,0.2122005671262741,4649.0,18,2025-12-19 15:46:31.482740,,,,,,,,, +0.27715983986854553,0.0037345513701438904,0.27715983986854553,4699.0,18,2025-12-19 15:55:06.572637,,,,,,,,, +,,,,18,2025-12-19 16:02:48.254829,0.25905686616897583,0.002874018158763647,0.25905686616897583,0.20303386449813843,0.003136978019028902,0.20303386449813843,,, +,,,,18,2025-12-19 16:03:00.851067,,,,,,,0.22596991062164307,0.003508082590997219,0.22596991062164307 +0.13280561566352844,0.0007645371952094138,0.13280561566352844,4749.0,19,2025-12-19 16:13:05.459541,,,,,,,,, +0.22932179272174835,0.00152356899343431,0.22932179272174835,4799.0,19,2025-12-19 16:22:37.051099,,,,,,,,, +0.31654098629951477,0.00212856219150126,0.31654098629951477,4849.0,19,2025-12-19 16:35:25.136434,,,,,,,,, +0.26714304089546204,0.0011560163693502545,0.26714304089546204,4899.0,19,2025-12-19 16:49:43.910703,,,,,,,,, +0.2802453935146332,0.0011417004279792309,0.2802453935146332,4949.0,19,2025-12-19 17:00:40.014330,,,,,,,,, +,,,,19,2025-12-19 17:09:12.452460,0.23053911328315735,0.002080062637105584,0.23053911328315735,0.19679279625415802,0.005508399568498135,0.19679279625415802,,, +,,,,19,2025-12-19 17:09:24.162042,,,,,,,0.24966558814048767,0.004523502662777901,0.24966558814048767 +0.363823801279068,0.003307297360152006,0.363823801279068,4999.0,20,2025-12-19 17:18:40.671035,,,,,,,,, +0.13287870585918427,0.0004908373230136931,0.13287870585918427,5049.0,20,2025-12-19 17:30:56.770580,,,,,,,,, +0.20445702970027924,0.0008128075278364122,0.20445702970027924,5099.0,20,2025-12-19 17:41:53.235368,,,,,,,,, +0.3546057939529419,0.0034894836135208607,0.3546057939529419,5149.0,20,2025-12-19 17:53:26.467717,,,,,,,,, +0.36615169048309326,0.017422206699848175,0.36615169048309326,5199.0,20,2025-12-19 18:09:21.250886,,,,,,,,, +,,,,20,2025-12-19 18:15:36.258690,0.20128922164440155,0.013167236000299454,0.20128922164440155,0.18372252583503723,0.002013958990573883,0.18372252583503723,,, +,,,,20,2025-12-19 18:15:57.809529,,,,,,,0.2349599003791809,0.003342988668009639,0.2349599003791809 +0.15608760714530945,0.0005359407514333725,0.15608760714530945,5249.0,21,2025-12-19 18:28:02.568207,,,,,,,,, +0.3427705764770508,0.005727624054998159,0.3427705764770508,5299.0,21,2025-12-19 18:38:17.494704,,,,,,,,, +0.153105229139328,0.0006215319153852761,0.153105229139328,5349.0,21,2025-12-19 18:48:44.854313,,,,,,,,, +0.03836352005600929,0.00013432840933091938,0.03836352005600929,5399.0,21,2025-12-19 18:57:44.809659,,,,,,,,, +0.2232373207807541,0.0011423188261687756,0.2232373207807541,5449.0,21,2025-12-19 19:13:23.457227,,,,,,,,, +,,,,21,2025-12-19 19:18:57.619785,0.22605308890342712,0.0027169520035386086,0.22605308890342712,0.1825030893087387,0.0010154227493330836,0.1825030893087387,,, +,,,,21,2025-12-19 19:19:19.455543,,,,,,,0.23529833555221558,0.003642433788627386,0.23529833555221558 +0.24506524205207825,0.0011724448995664716,0.24506524205207825,5499.0,22,2025-12-19 19:28:44.533544,,,,,,,,, +0.2147926241159439,0.0010824977653101087,0.2147926241159439,5549.0,22,2025-12-19 19:34:49.419393,,,,,,,,, +0.14520849287509918,0.0005852367612533271,0.14520849287509918,5599.0,22,2025-12-19 19:42:16.297012,,,,,,,,, +0.16682885587215424,0.0009144985233433545,0.16682885587215424,5649.0,22,2025-12-19 19:54:08.189482,,,,,,,,, +0.20695750415325165,0.0008168570348061621,0.20695750415325165,5699.0,22,2025-12-19 20:08:00.359462,,,,,,,,, +,,,,22,2025-12-19 20:13:32.224384,0.24902036786079407,0.003518250770866871,0.24902036786079407,0.2787752151489258,0.0062196520157158375,0.2787752151489258,,, +,,,,22,2025-12-19 20:13:43.765359,,,,,,,0.23596277832984924,0.003548246342688799,0.23596277832984924 +0.1961660385131836,0.0010300971334800124,0.1961660385131836,5749.0,23,2025-12-19 20:23:27.641634,,,,,,,,, +0.3286861479282379,0.004802646581083536,0.3286861479282379,5799.0,23,2025-12-19 20:40:22.852993,,,,,,,,, +0.2806589901447296,0.0015000681160017848,0.2806589901447296,5849.0,23,2025-12-19 20:50:42.850064,,,,,,,,, +0.1380346119403839,0.0005221018218435347,0.1380346119403839,5899.0,23,2025-12-19 20:58:30.887530,,,,,,,,, +0.24717746675014496,0.0009111024555750191,0.24717746675014496,5949.0,23,2025-12-19 21:09:45.927004,,,,,,,,, +,,,,23,2025-12-19 21:15:07.706640,0.23068208992481232,0.0036679187323898077,0.23068208992481232,0.2255161553621292,0.002794991247355938,0.2255161553621292,,, +,,,,23,2025-12-19 21:15:19.527860,,,,,,,0.2312050461769104,0.003935504704713821,0.2312050461769104 +0.1141032725572586,0.0005685696378350258,0.1141032725572586,5999.0,24,2025-12-19 21:23:51.622048,,,,,,,,, +0.2448895126581192,0.0026307515799999237,0.2448895126581192,6049.0,24,2025-12-19 21:36:44.838022,,,,,,,,, +0.3409416973590851,0.003733808873221278,0.3409416973590851,6099.0,24,2025-12-19 21:56:11.485120,,,,,,,,, +0.1678900420665741,0.0008372048032470047,0.1678900420665741,6149.0,24,2025-12-19 22:08:42.883448,,,,,,,,, +0.20669564604759216,0.0010232307249680161,0.20669564604759216,6199.0,24,2025-12-19 22:23:35.320321,,,,,,,,, +,,,,24,2025-12-19 22:28:46.740107,0.2187785506248474,0.0020808959379792213,0.2187785506248474,0.2235250025987625,0.0019378734286874533,0.2235250025987625,,, +,,,,24,2025-12-19 22:28:58.857029,,,,,,,0.22622792422771454,0.0038451666478067636,0.22622792422771454 +0.11122729629278183,0.0004037140461150557,0.11122729629278183,6249.0,25,2025-12-19 22:43:49.967236,,,,,,,,, +0.1788311004638672,0.0006384573644027114,0.1788311004638672,6299.0,25,2025-12-19 23:01:54.230160,,,,,,,,, +0.22189070284366608,0.0009050663211382926,0.22189070284366608,6349.0,25,2025-12-19 23:10:42.673191,,,,,,,,, +0.19340625405311584,0.000999037059955299,0.19340625405311584,6399.0,25,2025-12-19 23:20:19.724616,,,,,,,,, +,,,,25,2025-12-19 23:36:44.059011,0.21211941540241241,0.003030958818271756,0.21211941540241241,0.2304965704679489,0.004987283609807491,0.2304965704679489,,, +,,,,25,2025-12-19 23:36:55.965477,,,,,,,0.2264620065689087,0.0024823418352752924,0.2264620065689087 +0.3705788850784302,0.00289896409958601,0.3705788850784302,6449.0,26,2025-12-19 23:38:59.555108,,,,,,,,, +0.1437043994665146,0.0005463187699206173,0.1437043994665146,6499.0,26,2025-12-19 23:49:15.096628,,,,,,,,, +0.166590616106987,0.0006125041400082409,0.166590616106987,6549.0,26,2025-12-19 23:58:14.311150,,,,,,,,, +0.2365930825471878,0.0019355076365172863,0.2365930825471878,6599.0,26,2025-12-20 00:10:37.051340,,,,,,,,, +0.22682052850723267,0.0011373023735359311,0.22682052850723267,6649.0,26,2025-12-20 00:23:48.466111,,,,,,,,, +,,,,26,2025-12-20 00:39:46.684062,0.19869790971279144,0.0018699734937399626,0.19869790971279144,0.21880312263965607,0.001539085409604013,0.21880312263965607,,, +,,,,26,2025-12-20 00:39:59.692020,,,,,,,0.22602903842926025,0.003177021164447069,0.22602903842926025 +0.17568843066692352,0.0006899868603795767,0.17568843066692352,6699.0,27,2025-12-20 00:43:32.763985,,,,,,,,, +0.1464822143316269,0.0004931649309583008,0.1464822143316269,6749.0,27,2025-12-20 00:59:01.640176,,,,,,,,, +0.3082368075847626,0.0014461324317380786,0.3082368075847626,6799.0,27,2025-12-20 01:07:34.440616,,,,,,,,, +0.197021022439003,0.000880956940818578,0.197021022439003,6849.0,27,2025-12-20 01:22:07.808599,,,,,,,,, +0.1280541718006134,0.0007357692811638117,0.1280541718006134,6899.0,27,2025-12-20 01:29:41.088033,,,,,,,,, +,,,,27,2025-12-20 01:41:51.515278,0.2216881960630417,0.0028012078255414963,0.2216881960630417,0.1769588142633438,0.0017844706308096647,0.1769588142633438,,, +,,,,27,2025-12-20 01:42:12.734167,,,,,,,0.21888622641563416,0.002900495892390609,0.21888622641563416 +0.04377826675772667,0.00015772074402775615,0.04377826675772667,6949.0,28,2025-12-20 01:45:49.414044,,,,,,,,, +0.3820183575153351,0.0023546761367470026,0.3820183575153351,6999.0,28,2025-12-20 01:59:49.400001,,,,,,,,, +0.23657049238681793,0.0024438516702502966,0.23657049238681793,7049.0,28,2025-12-20 02:12:00.498340,,,,,,,,, +0.17581520974636078,0.0006285826093517244,0.17581520974636078,7099.0,28,2025-12-20 02:25:44.052782,,,,,,,,, +0.18726277351379395,0.0010832150001078844,0.18726277351379395,7149.0,28,2025-12-20 02:38:21.628864,,,,,,,,, +,,,,28,2025-12-20 02:53:29.744366,0.277013897895813,0.01040742639452219,0.277013897895813,0.20560061931610107,0.0014080486726015806,0.20560061931610107,,, +,,,,28,2025-12-20 02:53:41.458695,,,,,,,0.22835548222064972,0.004334555007517338,0.22835548222064972 +0.11738140881061554,0.0004825775104109198,0.11738140881061554,7199.0,29,2025-12-20 02:58:02.165314,,,,,,,,, +0.22772261500358582,0.001115567283704877,0.22772261500358582,7249.0,29,2025-12-20 03:08:53.381491,,,,,,,,, +0.3042204678058624,0.001664145034737885,0.3042204678058624,7299.0,29,2025-12-20 03:23:49.059109,,,,,,,,, +0.2598176598548889,0.0029425511602312326,0.2598176598548889,7349.0,29,2025-12-20 03:32:12.036070,,,,,,,,, +0.138965904712677,0.001904923003166914,0.138965904712677,7399.0,29,2025-12-20 03:45:48.843163,,,,,,,,, +,,,,29,2025-12-20 04:01:28.948146,0.16970178484916687,0.001036516739986837,0.16970178484916687,0.2477426826953888,0.004533721599727869,0.2477426826953888,,, +,,,,29,2025-12-20 04:01:40.491534,,,,,,,0.2316180169582367,0.004054228775203228,0.2316180169582367 +0.17428678274154663,0.00142040872015059,0.17428678274154663,7449.0,30,2025-12-20 04:07:56.885652,,,,,,,,, +0.4210222065448761,0.0019582335371524096,0.4210222065448761,7499.0,30,2025-12-20 04:18:02.378230,,,,,,,,, +0.308363139629364,0.0017703620251268148,0.308363139629364,7549.0,30,2025-12-20 04:30:42.500481,,,,,,,,, +0.13626490533351898,0.000694898481015116,0.13626490533351898,7599.0,30,2025-12-20 04:42:32.814542,,,,,,,,, +0.3408443331718445,0.0016880177427083254,0.3408443331718445,7649.0,30,2025-12-20 04:51:22.447790,,,,,,,,, +,,,,30,2025-12-20 05:05:57.688529,0.19005027413368225,0.003961995709687471,0.19005027413368225,0.23490825295448303,0.00970070343464613,0.23490825295448303,,, +,,,,30,2025-12-20 05:06:09.675070,,,,,,,0.22102324664592743,0.0036179774906486273,0.22102324664592743 +0.08303981274366379,0.00029834272572770715,0.08303981274366379,7699.0,31,2025-12-20 05:10:20.523497,,,,,,,,, +0.2233649045228958,0.0009084050543606281,0.2233649045228958,7749.0,31,2025-12-20 05:21:35.014890,,,,,,,,, +0.2117622047662735,0.001141474349424243,0.2117622047662735,7799.0,31,2025-12-20 05:32:10.101046,,,,,,,,, +0.054197296500205994,0.00019870043615810573,0.054197296500205994,7849.0,31,2025-12-20 05:42:51.856650,,,,,,,,, +0.2699034512042999,0.0015741263050585985,0.2699034512042999,7899.0,31,2025-12-20 05:53:23.617144,,,,,,,,, +,,,,31,2025-12-20 06:05:42.214766,0.18942996859550476,0.0025369422510266304,0.18942996859550476,0.23946677148342133,0.013558662496507168,0.23946677148342133,,, +,,,,31,2025-12-20 06:05:53.782377,,,,,,,0.22225573658943176,0.0042383791878819466,0.22225573658943176 +0.2147854119539261,0.000813933729659766,0.2147854119539261,7949.0,32,2025-12-20 06:12:56.829915,,,,,,,,, +0.18300983309745789,0.0006935088895261288,0.18300983309745789,7999.0,32,2025-12-20 06:20:29.688721,,,,,,,,, +0.13459911942481995,0.0004492403822951019,0.13459911942481995,8049.0,32,2025-12-20 06:33:16.110409,,,,,,,,, +0.17352227866649628,0.0007545159314759076,0.17352227866649628,8099.0,32,2025-12-20 06:42:26.444329,,,,,,,,, +0.09526587277650833,0.00041492836317047477,0.09526587277650833,8149.0,32,2025-12-20 06:51:19.240892,,,,,,,,, +,,,,32,2025-12-20 07:05:59.265163,0.20381788909435272,0.0037709102034568787,0.20381788909435272,0.2115406095981598,0.0023343514185398817,0.2115406095981598,,, +,,,,32,2025-12-20 07:06:10.486882,,,,,,,0.23033593595027924,0.004080940969288349,0.23033593595027924 +0.40923839807510376,0.004166199825704098,0.40923839807510376,8199.0,33,2025-12-20 07:13:24.132857,,,,,,,,, +0.3246418535709381,0.007406461983919144,0.3246418535709381,8249.0,33,2025-12-20 07:32:38.172531,,,,,,,,, +0.25537535548210144,0.001435760292224586,0.25537535548210144,8299.0,33,2025-12-20 07:41:24.851520,,,,,,,,, +0.30370455980300903,0.006453062407672405,0.30370455980300903,8349.0,33,2025-12-20 07:50:48.600944,,,,,,,,, +0.05529129132628441,0.0001980063389055431,0.05529129132628441,8399.0,33,2025-12-20 08:03:30.918228,,,,,,,,, +,,,,33,2025-12-20 08:12:58.444679,0.17628513276576996,0.0016973518067970872,0.17628513276576996,0.2063741832971573,0.008406569249927998,0.2063741832971573,,, +,,,,33,2025-12-20 08:13:10.019999,,,,,,,0.2288467288017273,0.004151416476815939,0.2288467288017273 +0.4277067184448242,0.0037319695111364126,0.4277067184448242,8449.0,34,2025-12-20 08:18:12.633924,,,,,,,,, +0.15278516709804535,0.0005268083186820149,0.15278516709804535,8499.0,34,2025-12-20 08:28:19.843976,,,,,,,,, +0.2713406980037689,0.003755454206839204,0.2713406980037689,8549.0,34,2025-12-20 08:38:37.827022,,,,,,,,, +0.2696228325366974,0.0018392630154266953,0.2696228325366974,8599.0,34,2025-12-20 08:53:09.894864,,,,,,,,, +0.25091230869293213,0.0025078977923840284,0.25091230869293213,8649.0,34,2025-12-20 09:00:14.173068,,,,,,,,, +,,,,34,2025-12-20 09:13:34.522347,0.23587217926979065,0.00236429157666862,0.23587217926979065,0.2252148538827896,0.0023918072693049908,0.2252148538827896,,, +,,,,34,2025-12-20 09:13:46.460024,,,,,,,0.2237417846918106,0.002738330978900194,0.2237417846918106 +0.3177798092365265,0.0015723335091024637,0.3177798092365265,8699.0,35,2025-12-20 09:19:32.262583,,,,,,,,, +0.2677772641181946,0.0034095747396349907,0.2677772641181946,8749.0,35,2025-12-20 09:26:57.976950,,,,,,,,, +0.18624247610569,0.0010328347561880946,0.18624247610569,8799.0,35,2025-12-20 09:36:30.421564,,,,,,,,, +0.32717689871788025,0.0034113856963813305,0.32717689871788025,8849.0,35,2025-12-20 09:45:23.313995,,,,,,,,, +0.2537536323070526,0.0012900748988613486,0.2537536323070526,8899.0,35,2025-12-20 09:55:29.002575,,,,,,,,, +,,,,35,2025-12-20 10:06:38.065854,0.25372350215911865,0.010084976442158222,0.25372350215911865,0.21949806809425354,0.004684359300881624,0.21949806809425354,,, +,,,,35,2025-12-20 10:06:49.941521,,,,,,,0.2359435260295868,0.0035652685910463333,0.2359435260295868 +0.17356432974338531,0.005649680737406015,0.17356432974338531,8949.0,36,2025-12-20 10:12:09.994419,,,,,,,,, +0.34731611609458923,0.008099351078271866,0.34731611609458923,8999.0,36,2025-12-20 10:21:09.742921,,,,,,,,, +0.601898729801178,0.025755412876605988,0.601898729801178,9049.0,36,2025-12-20 10:30:53.179190,,,,,,,,, +0.45566749572753906,0.003551705740392208,0.45566749572753906,9099.0,36,2025-12-20 10:43:53.421332,,,,,,,,, +0.12875908613204956,0.0004449347616173327,0.12875908613204956,9149.0,36,2025-12-20 10:56:34.691692,,,,,,,,, +,,,,36,2025-12-20 11:06:17.405974,0.23110298812389374,0.008989437483251095,0.23110298812389374,0.19698834419250488,0.004664834588766098,0.19698834419250488,,, +,,,,36,2025-12-20 11:06:29.220052,,,,,,,0.23027102649211884,0.0034434671979397535,0.23027102649211884 +0.32616671919822693,0.001751597854308784,0.32616671919822693,9199.0,37,2025-12-20 11:11:41.968500,,,,,,,,, +0.20390470325946808,0.0007688156911171973,0.20390470325946808,9249.0,37,2025-12-20 11:19:33.283565,,,,,,,,, +0.2610064744949341,0.0012746121501550078,0.2610064744949341,9299.0,37,2025-12-20 11:34:32.678994,,,,,,,,, +0.1712213158607483,0.0035016590263694525,0.1712213158607483,9349.0,37,2025-12-20 11:45:57.933145,,,,,,,,, +0.23942717909812927,0.0024238047190010548,0.23942717909812927,9399.0,37,2025-12-20 11:58:41.655109,,,,,,,,, +,,,,37,2025-12-20 12:06:21.176811,0.19330976903438568,0.0031282261479645967,0.19330976903438568,0.24963761866092682,0.0036654856521636248,0.24963761866092682,,, +,,,,37,2025-12-20 12:06:33.066623,,,,,,,0.2282266467809677,0.005017879419028759,0.2282266467809677 +0.1532805860042572,0.0007048137485980988,0.1532805860042572,9449.0,38,2025-12-20 12:14:32.275366,,,,,,,,, +0.2120356261730194,0.0007527799461968243,0.2120356261730194,9499.0,38,2025-12-20 12:23:41.098815,,,,,,,,, +0.17049561440944672,0.0005932340864092112,0.17049561440944672,9549.0,38,2025-12-20 12:32:38.915479,,,,,,,,, +0.2890915870666504,0.0015875366516411304,0.2890915870666504,9599.0,38,2025-12-20 12:45:33.813504,,,,,,,,, +0.2407921403646469,0.0011084612924605608,0.2407921403646469,9649.0,38,2025-12-20 12:56:15.312447,,,,,,,,, +,,,,38,2025-12-20 13:04:20.001722,0.21121443808078766,0.0016866882797330618,0.21121443808078766,0.21748332679271698,0.0028483455535024405,0.21748332679271698,,, +,,,,38,2025-12-20 13:04:31.659191,,,,,,,0.23301094770431519,0.0035301402676850557,0.23301094770431519 +0.44406166672706604,0.003509603440761566,0.44406166672706604,9699.0,39,2025-12-20 13:11:26.497611,,,,,,,,, +0.37401822209358215,0.0025761921424418688,0.37401822209358215,9749.0,39,2025-12-20 13:23:42.722381,,,,,,,,, +0.13334178924560547,0.00057392125017941,0.13334178924560547,9799.0,39,2025-12-20 13:38:20.928061,,,,,,,,, +0.06677360087633133,0.00024291910813190043,0.06677360087633133,9849.0,39,2025-12-20 13:53:26.187776,,,,,,,,, +0.19489163160324097,0.000702083867508918,0.19489163160324097,9899.0,39,2025-12-20 14:03:36.254766,,,,,,,,, +,,,,39,2025-12-20 14:12:59.097941,0.2129673808813095,0.0012801551492884755,0.2129673808813095,0.2128048837184906,0.003184016328305006,0.2128048837184906,,, +,,,,39,2025-12-20 14:13:10.981435,,,,,,,0.2336161881685257,0.00435979338362813,0.2336161881685257 +0.2104814350605011,0.001222022925503552,0.2104814350605011,9949.0,40,2025-12-20 14:26:05.358902,,,,,,,,, +0.30553415417671204,0.0015616531018167734,0.30553415417671204,9999.0,40,2025-12-20 14:33:36.018151,,,,,,,,, +0.4811691343784332,0.005348658189177513,0.4811691343784332,10049.0,40,2025-12-20 14:48:11.180763,,,,,,,,, +0.3959956467151642,0.002643626881763339,0.3959956467151642,10099.0,40,2025-12-20 14:55:22.155279,,,,,,,,, +0.23075036704540253,0.001452116179279983,0.23075036704540253,10149.0,40,2025-12-20 15:11:20.978352,,,,,,,,, +,,,,40,2025-12-20 15:21:56.600623,0.2008352428674698,0.001862423843704164,0.2008352428674698,0.23047451674938202,0.009356741793453693,0.23047451674938202,,, +,,,,40,2025-12-20 15:22:09.002482,,,,,,,0.2312745451927185,0.00423929188400507,0.2312745451927185 +0.15368737280368805,0.000571667798794806,0.15368737280368805,10199.0,41,2025-12-20 15:31:50.150399,,,,,,,,, +0.1843881458044052,0.000769151549320668,0.1843881458044052,10249.0,41,2025-12-20 15:44:39.327856,,,,,,,,, +0.1078491061925888,0.0003728547890204936,0.1078491061925888,10299.0,41,2025-12-20 15:54:47.476789,,,,,,,,, +0.3613307476043701,0.001986506162211299,0.3613307476043701,10349.0,41,2025-12-20 16:07:51.901670,,,,,,,,, +0.4703223407268524,0.0234546959400177,0.4703223407268524,10399.0,41,2025-12-20 16:23:26.104922,,,,,,,,, +,,,,41,2025-12-20 16:32:13.426432,0.21722030639648438,0.0018873265944421291,0.21722030639648438,0.23760093748569489,0.005868187639862299,0.23760093748569489,,, +,,,,41,2025-12-20 16:32:25.300468,,,,,,,0.2280288189649582,0.004143727943301201,0.2280288189649582 +0.36679986119270325,0.01700458861887455,0.36679986119270325,10449.0,42,2025-12-20 16:44:08.068833,,,,,,,,, +0.30902335047721863,0.0018084150506183505,0.30902335047721863,10499.0,42,2025-12-20 16:54:02.933245,,,,,,,,, +0.530544638633728,0.00796428881585598,0.530544638633728,10549.0,42,2025-12-20 17:05:30.550636,,,,,,,,, +0.23588170111179352,0.0012304279953241348,0.23588170111179352,10599.0,42,2025-12-20 17:14:46.005226,,,,,,,,, +0.29416751861572266,0.0011581663275137544,0.29416751861572266,10649.0,42,2025-12-20 17:24:15.350352,,,,,,,,, +,,,,42,2025-12-20 17:31:07.734819,0.23535138368606567,0.0018557233270257711,0.23535138368606567,0.20955468714237213,0.0015798714011907578,0.20955468714237213,,, +,,,,42,2025-12-20 17:31:19.450429,,,,,,,0.23770803213119507,0.004673898220062256,0.23770803213119507 +0.5160359740257263,0.0033641287591308355,0.5160359740257263,10699.0,43,2025-12-20 17:44:23.914156,,,,,,,,, +0.11972654610872269,0.0004006398667115718,0.11972654610872269,10749.0,43,2025-12-20 17:52:35.436859,,,,,,,,, +0.1731649935245514,0.0006356409285217524,0.1731649935245514,10799.0,43,2025-12-20 18:07:56.278120,,,,,,,,, +0.14686034619808197,0.0005268746754154563,0.14686034619808197,10849.0,43,2025-12-20 18:16:50.530808,,,,,,,,, +0.2618703246116638,0.0012435347307473421,0.2618703246116638,10899.0,43,2025-12-20 18:24:44.390954,,,,,,,,, +,,,,43,2025-12-20 18:30:50.885610,0.22340676188468933,0.00201800256036222,0.22340676188468933,0.219914972782135,0.004536184016615152,0.219914972782135,,, +,,,,43,2025-12-20 18:31:03.323817,,,,,,,0.22853483259677887,0.00418302183970809,0.22853483259677887 +0.3059520125389099,0.0012342684203758836,0.3059520125389099,10949.0,44,2025-12-20 18:40:16.499690,,,,,,,,, +0.1991233080625534,0.0010027455864474177,0.1991233080625534,10999.0,44,2025-12-20 18:52:19.390506,,,,,,,,, +0.04494449123740196,0.0001633733045309782,0.04494449123740196,11049.0,44,2025-12-20 19:02:13.089778,,,,,,,,, +0.2838501036167145,0.0012122909538447857,0.2838501036167145,11099.0,44,2025-12-20 19:08:26.618969,,,,,,,,, +0.050371717661619186,0.0001840270560933277,0.050371717661619186,11149.0,44,2025-12-20 19:17:09.234842,,,,,,,,, +,,,,44,2025-12-20 19:22:52.950476,0.19732171297073364,0.0014255797723308206,0.19732171297073364,0.22768864035606384,0.003028744598850608,0.22768864035606384,,, +,,,,44,2025-12-20 19:23:05.071605,,,,,,,0.2296190857887268,0.002646777080371976,0.2296190857887268 +0.04522102698683739,0.0001690554345259443,0.04522102698683739,11199.0,45,2025-12-20 19:30:43.911097,,,,,,,,, +0.20582905411720276,0.0007814503042027354,0.20582905411720276,11249.0,45,2025-12-20 19:43:55.016186,,,,,,,,, +0.22471627593040466,0.040592703968286514,0.22471627593040466,11299.0,45,2025-12-20 19:53:13.805367,,,,,,,,, +0.11511664092540741,0.00040520893526263535,0.11511664092540741,11349.0,45,2025-12-20 20:03:32.404334,,,,,,,,, +0.2621143162250519,0.002005775924772024,0.2621143162250519,11399.0,45,2025-12-20 20:10:31.593482,,,,,,,,, +,,,,45,2025-12-20 20:18:28.576486,0.25154098868370056,0.0023704087361693382,0.25154098868370056,0.227385014295578,0.002728157676756382,0.227385014295578,,, +,,,,45,2025-12-20 20:18:40.446545,,,,,,,0.22137950360774994,0.0038901236839592457,0.22137950360774994 +0.3446832299232483,0.0051575759425759315,0.3446832299232483,11449.0,46,2025-12-20 20:31:10.776562,,,,,,,,, +0.15539130568504333,0.0005176174454391003,0.15539130568504333,11499.0,46,2025-12-20 20:39:46.665773,,,,,,,,, +0.18153762817382812,0.0006890846998430789,0.18153762817382812,11549.0,46,2025-12-20 20:49:14.020673,,,,,,,,, +0.3440053164958954,0.006049383897334337,0.3440053164958954,11599.0,46,2025-12-20 21:03:52.548733,,,,,,,,, +0.3076738715171814,0.0022661075927317142,0.3076738715171814,11649.0,46,2025-12-20 21:14:31.265721,,,,,,,,, +,,,,46,2025-12-20 21:19:57.109511,0.22922660410404205,0.0024514200631529093,0.22922660410404205,0.2009488344192505,0.0023385160602629185,0.2009488344192505,,, +,,,,46,2025-12-20 21:20:09.341593,,,,,,,0.23400288820266724,0.0032950183376669884,0.23400288820266724 +0.13064759969711304,0.0005753138102591038,0.13064759969711304,11699.0,47,2025-12-20 21:35:59.722303,,,,,,,,, +0.28461506962776184,0.0011029966408386827,0.28461506962776184,11749.0,47,2025-12-20 21:48:06.672383,,,,,,,,, +0.2737623155117035,0.001715439255349338,0.2737623155117035,11799.0,47,2025-12-20 21:57:17.530131,,,,,,,,, +0.1378563940525055,0.000622218067292124,0.1378563940525055,11849.0,47,2025-12-20 22:05:05.550686,,,,,,,,, +0.2384559065103531,0.0011811940930783749,0.2384559065103531,11899.0,47,2025-12-20 22:13:27.601904,,,,,,,,, +,,,,47,2025-12-20 22:18:43.435031,0.20880578458309174,0.011883552186191082,0.20880578458309174,0.1800176054239273,0.0014979810221120715,0.1800176054239273,,, +,,,,47,2025-12-20 22:19:05.855342,,,,,,,0.22198709845542908,0.004343494772911072,0.22198709845542908 +0.08936936408281326,0.0003621706855483353,0.08936936408281326,11949.0,48,2025-12-20 22:34:01.572165,,,,,,,,, +0.11577708274126053,0.00039246518281288445,0.11577708274126053,11999.0,48,2025-12-20 22:42:54.548374,,,,,,,,, +0.19049273431301117,0.0008716837619431317,0.19049273431301117,12049.0,48,2025-12-20 22:57:11.033907,,,,,,,,, +0.2706436812877655,0.0032223365269601345,0.2706436812877655,12099.0,48,2025-12-20 23:06:02.644974,,,,,,,,, +0.1304132044315338,0.000456315086921677,0.1304132044315338,12149.0,48,2025-12-20 23:17:10.463038,,,,,,,,, +,,,,48,2025-12-20 23:22:19.592639,0.21852494776248932,0.002072057221084833,0.21852494776248932,0.21007327735424042,0.005672391504049301,0.21007327735424042,,, +,,,,48,2025-12-20 23:22:31.389179,,,,,,,0.2218552529811859,0.0032996488735079765,0.2218552529811859 +0.16595982015132904,0.0005889741005375981,0.16595982015132904,12199.0,49,2025-12-20 23:32:05.318916,,,,,,,,, +0.11693183332681656,0.00046095054131001234,0.11693183332681656,12249.0,49,2025-12-20 23:45:11.297036,,,,,,,,, +0.14001692831516266,0.0019205642165616155,0.14001692831516266,12299.0,49,2025-12-20 23:54:32.230133,,,,,,,,, +0.20222584903240204,0.0009360871044918895,0.20222584903240204,12349.0,49,2025-12-21 00:09:14.057807,,,,,,,,, +0.18693822622299194,0.0007534162723459303,0.18693822622299194,12399.0,49,2025-12-21 00:22:05.109189,,,,,,,,, +,,,,49,2025-12-21 00:27:03.187605,0.20377448201179504,0.008210709318518639,0.20377448201179504,0.22077861428260803,0.008646508678793907,0.22077861428260803,,, +,,,,49,2025-12-21 00:27:14.999755,,,,,,,0.2285233736038208,0.0030740664806216955,0.2285233736038208 +0.23838819563388824,0.0012939042644575238,0.23838819563388824,12449.0,50,2025-12-21 00:43:56.955662,,,,,,,,, +0.4964311718940735,0.005235353019088507,0.4964311718940735,12499.0,50,2025-12-21 00:54:54.443057,,,,,,,,, +0.25904083251953125,0.005905487574636936,0.25904083251953125,12549.0,50,2025-12-21 01:05:18.730630,,,,,,,,, +0.1694335639476776,0.00072688446380198,0.1694335639476776,12599.0,50,2025-12-21 01:13:11.728897,,,,,,,,, +,,,,50,2025-12-21 01:32:38.166563,0.17231903970241547,0.0022341543808579445,0.17231903970241547,0.2322893589735031,0.0023090168833732605,0.2322893589735031,,, +,,,,50,2025-12-21 01:32:49.829294,,,,,,,0.23959703743457794,0.0043349419720470905,0.23959703743457794 +0.26044201850891113,0.0025641564279794693,0.26044201850891113,12649.0,51,2025-12-21 01:33:45.353857,,,,,,,,, +0.28983160853385925,0.0014746985398232937,0.28983160853385925,12699.0,51,2025-12-21 01:44:41.219337,,,,,,,,, +0.20278288424015045,0.0009440963622182608,0.20278288424015045,12749.0,51,2025-12-21 01:53:07.641909,,,,,,,,, +0.05487879365682602,0.00019952574803028256,0.05487879365682602,12799.0,51,2025-12-21 02:02:56.733135,,,,,,,,, +0.20205558836460114,0.0018446524627506733,0.20205558836460114,12849.0,51,2025-12-21 02:11:20.381387,,,,,,,,, +,,,,51,2025-12-21 02:24:02.827032,0.21522678434848785,0.003940907306969166,0.21522678434848785,0.20015716552734375,0.002481566509231925,0.20015716552734375,,, +,,,,51,2025-12-21 02:24:14.924980,,,,,,,0.22289682924747467,0.0027423822320997715,0.22289682924747467 +0.10129362344741821,0.00033858773531392217,0.10129362344741821,12899.0,52,2025-12-21 02:26:20.387104,,,,,,,,, +0.1222376823425293,0.00045092790969647467,0.1222376823425293,12949.0,52,2025-12-21 02:35:40.746102,,,,,,,,, +0.03301176056265831,0.00013018901518080384,0.03301176056265831,12999.0,52,2025-12-21 02:46:06.597583,,,,,,,,, +0.14801664650440216,0.0005104339215904474,0.14801664650440216,13049.0,52,2025-12-21 02:58:22.760250,,,,,,,,, +0.12169872969388962,0.0004084506363142282,0.12169872969388962,13099.0,52,2025-12-21 03:12:15.404101,,,,,,,,, +,,,,52,2025-12-21 03:28:08.873523,0.21841803193092346,0.002860557520762086,0.21841803193092346,0.17882609367370605,0.001354198670014739,0.17882609367370605,,, +,,,,52,2025-12-21 03:28:30.762929,,,,,,,0.2249927669763565,0.003711593570187688,0.2249927669763565 +0.1214384213089943,0.0004100178193766624,0.1214384213089943,13149.0,53,2025-12-21 03:31:33.656477,,,,,,,,, +0.19333113729953766,0.000767382385674864,0.19333113729953766,13199.0,53,2025-12-21 03:41:04.801158,,,,,,,,, +0.22249867022037506,0.0007954289321787655,0.22249867022037506,13249.0,53,2025-12-21 03:53:48.512231,,,,,,,,, +0.16426749527454376,0.000575867947191,0.16426749527454376,13299.0,53,2025-12-21 04:04:39.323303,,,,,,,,, +0.39757177233695984,0.0029069131705909967,0.39757177233695984,13349.0,53,2025-12-21 04:15:04.631718,,,,,,,,, +,,,,53,2025-12-21 04:27:29.282502,0.21408453583717346,0.0031754877418279648,0.21408453583717346,0.18923436105251312,0.00197509303689003,0.18923436105251312,,, +,,,,53,2025-12-21 04:27:41.774063,,,,,,,0.22434040904045105,0.0028966953977942467,0.22434040904045105 +0.15690001845359802,0.0005263159400783479,0.15690001845359802,13399.0,54,2025-12-21 04:31:19.119656,,,,,,,,, +0.10777979344129562,0.0003709054144565016,0.10777979344129562,13449.0,54,2025-12-21 04:39:34.015558,,,,,,,,, +0.28780150413513184,0.0014868028229102492,0.28780150413513184,13499.0,54,2025-12-21 04:52:49.261978,,,,,,,,, +0.2774384915828705,0.001084899646230042,0.2774384915828705,13549.0,54,2025-12-21 05:02:55.998020,,,,,,,,, +0.29150813817977905,0.0015413530636578798,0.29150813817977905,13599.0,54,2025-12-21 05:11:30.378767,,,,,,,,, +,,,,54,2025-12-21 05:23:41.246359,0.21574988961219788,0.003998427651822567,0.21574988961219788,0.22896626591682434,0.008435724303126335,0.22896626591682434,,, +,,,,54,2025-12-21 05:23:53.347154,,,,,,,0.22830873727798462,0.00407053716480732,0.22830873727798462 +0.05209231376647949,0.00018589147657621652,0.05209231376647949,13649.0,55,2025-12-21 05:27:36.198645,,,,,,,,, +0.10977043956518173,0.00046900392044335604,0.10977043956518173,13699.0,55,2025-12-21 05:43:53.545474,,,,,,,,, +0.263853520154953,0.0015365061117336154,0.263853520154953,13749.0,55,2025-12-21 05:52:20.297541,,,,,,,,, +0.20414739847183228,0.0007110672304406762,0.20414739847183228,13799.0,55,2025-12-21 06:04:29.835678,,,,,,,,, +0.13355767726898193,0.0005444384296424687,0.13355767726898193,13849.0,55,2025-12-21 06:14:47.110228,,,,,,,,, +,,,,55,2025-12-21 06:28:25.005178,0.1732005625963211,0.0009573952411301434,0.1732005625963211,0.2110338658094406,0.0023128429893404245,0.2110338658094406,,, +,,,,55,2025-12-21 06:28:37.131612,,,,,,,0.233661487698555,0.003155204001814127,0.233661487698555 +0.5910272002220154,0.00811945553869009,0.5910272002220154,13899.0,56,2025-12-21 06:32:43.804476,,,,,,,,, +0.397538423538208,0.002276567043736577,0.397538423538208,13949.0,56,2025-12-21 06:44:21.343164,,,,,,,,, +0.22296041250228882,0.0013013883726671338,0.22296041250228882,13999.0,56,2025-12-21 06:53:31.546413,,,,,,,,, +0.27146485447883606,0.0015594128053635359,0.27146485447883606,14049.0,56,2025-12-21 07:00:54.337575,,,,,,,,, +0.25281867384910583,0.009257453493773937,0.25281867384910583,14099.0,56,2025-12-21 07:13:55.662691,,,,,,,,, +,,,,56,2025-12-21 07:23:50.019810,0.18984463810920715,0.005759696010500193,0.18984463810920715,0.23799891769886017,0.001975554972887039,0.23799891769886017,,, +,,,,56,2025-12-21 07:24:02.800727,,,,,,,0.22811578214168549,0.0031857029534876347,0.22811578214168549 +0.3284502625465393,0.0021881989669054747,0.3284502625465393,14149.0,57,2025-12-21 07:27:43.600553,,,,,,,,, +0.27786630392074585,0.002018038649111986,0.27786630392074585,14199.0,57,2025-12-21 07:34:39.308242,,,,,,,,, +0.3061753511428833,0.0029001960065215826,0.3061753511428833,14249.0,57,2025-12-21 07:42:55.598211,,,,,,,,, +0.5506630539894104,0.010222313925623894,0.5506630539894104,14299.0,57,2025-12-21 07:51:14.943167,,,,,,,,, +0.1309284120798111,0.00046125284279696643,0.1309284120798111,14349.0,57,2025-12-21 08:07:59.668106,,,,,,,,, +,,,,57,2025-12-21 08:17:12.054607,0.25612694025039673,0.0023827513214200735,0.25612694025039673,0.1971760243177414,0.001662130351178348,0.1971760243177414,,, +,,,,57,2025-12-21 08:17:24.154688,,,,,,,0.23622362315654755,0.004133310168981552,0.23622362315654755 +0.032775864005088806,0.0001340454036835581,0.032775864005088806,14399.0,58,2025-12-21 08:24:36.406925,,,,,,,,, +0.5550662875175476,0.00983305461704731,0.5550662875175476,14449.0,58,2025-12-21 08:30:44.321659,,,,,,,,, +0.3669663071632385,0.0021861533168703318,0.3669663071632385,14499.0,58,2025-12-21 08:42:33.099860,,,,,,,,, +0.21174153685569763,0.00071804627077654,0.21174153685569763,14549.0,58,2025-12-21 08:56:42.424630,,,,,,,,, +0.3050215244293213,0.0022514229640364647,0.3050215244293213,14599.0,58,2025-12-21 09:08:58.081696,,,,,,,,, +,,,,58,2025-12-21 09:19:18.502257,0.22152256965637207,0.009362194687128067,0.22152256965637207,0.1894080489873886,0.0012141289189457893,0.1894080489873886,,, +,,,,58,2025-12-21 09:19:31.033439,,,,,,,0.21280081570148468,0.0030957201961427927,0.21280081570148468 +0.12083760648965836,0.00046091366675682366,0.12083760648965836,14649.0,59,2025-12-21 09:28:18.149566,,,,,,,,, +0.42788633704185486,0.005223172716796398,0.42788633704185486,14699.0,59,2025-12-21 09:37:23.411351,,,,,,,,, +0.13666750490665436,0.0004738967982120812,0.13666750490665436,14749.0,59,2025-12-21 09:54:59.679320,,,,,,,,, +0.30406203866004944,0.002324227010831237,0.30406203866004944,14799.0,59,2025-12-21 10:01:30.796619,,,,,,,,, +0.39749985933303833,0.0031396655831485987,0.39749985933303833,14849.0,59,2025-12-21 10:16:12.960078,,,,,,,,, +,,,,59,2025-12-21 10:25:28.582025,0.18104276061058044,0.0012222733348608017,0.18104276061058044,0.1929473727941513,0.00843797530978918,0.1929473727941513,,, +,,,,59,2025-12-21 10:25:40.234608,,,,,,,0.2344837784767151,0.003651241771876812,0.2344837784767151 +0.3740682601928711,0.002260494977235794,0.3740682601928711,14899.0,60,2025-12-21 10:33:28.717146,,,,,,,,, +0.3673565685749054,0.0021682740189135075,0.3673565685749054,14949.0,60,2025-12-21 10:44:56.061547,,,,,,,,, +0.2960169315338135,0.02306360751390457,0.2960169315338135,14999.0,60,2025-12-21 10:54:02.924724,,,,,,,,, +0.17171627283096313,0.004705514758825302,0.17171627283096313,15049.0,60,2025-12-21 11:07:20.495019,,,,,,,,, +0.18541380763053894,0.0007272888906300068,0.18541380763053894,15099.0,60,2025-12-21 11:22:21.700109,,,,,,,,, +,,,,60,2025-12-21 11:32:24.717894,0.2542646527290344,0.005274586379528046,0.2542646527290344,0.20519348978996277,0.002408848609775305,0.20519348978996277,,, +,,,,60,2025-12-21 11:32:36.738782,,,,,,,0.2219548523426056,0.0037871862296015024,0.2219548523426056 +0.08857116103172302,0.00030724002863280475,0.08857116103172302,15149.0,61,2025-12-21 11:39:31.573217,,,,,,,,, +0.2814491093158722,0.0017730138497427106,0.2814491093158722,15199.0,61,2025-12-21 11:52:12.954785,,,,,,,,, +0.260145366191864,0.001575613277964294,0.260145366191864,15249.0,61,2025-12-21 12:00:46.785578,,,,,,,,, +0.1925215870141983,0.0006738573429174721,0.1925215870141983,15299.0,61,2025-12-21 12:13:14.002111,,,,,,,,, +0.13941894471645355,0.00047037279000505805,0.13941894471645355,15349.0,61,2025-12-21 12:20:14.670073,,,,,,,,, +,,,,61,2025-12-21 12:28:40.721311,0.22646531462669373,0.00334328506141901,0.22646531462669373,0.1640632152557373,0.0011976177338510752,0.1640632152557373,,, +,,,,61,2025-12-21 12:29:03.262603,,,,,,,0.22551503777503967,0.004659491125494242,0.22551503777503967 +0.25149571895599365,0.0010844103526324034,0.25149571895599365,15399.0,62,2025-12-21 12:37:52.130061,,,,,,,,, +0.26221051812171936,0.0011072158813476562,0.26221051812171936,15449.0,62,2025-12-21 12:46:16.812718,,,,,,,,, +0.03349992260336876,0.00012958384468220174,0.03349992260336876,15499.0,62,2025-12-21 12:55:00.901949,,,,,,,,, +0.19873172044754028,0.000886601977981627,0.19873172044754028,15549.0,62,2025-12-21 13:04:33.215705,,,,,,,,, +0.0906672328710556,0.0004635299846995622,0.0906672328710556,15599.0,62,2025-12-21 13:11:29.729281,,,,,,,,, +,,,,62,2025-12-21 13:21:52.537882,0.20856061577796936,0.01320649404078722,0.20856061577796936,0.20641103386878967,0.0017802397487685084,0.20641103386878967,,, +,,,,62,2025-12-21 13:22:05.490888,,,,,,,0.23395276069641113,0.0032053401228040457,0.23395276069641113 +0.27605414390563965,0.0015603861538693309,0.27605414390563965,15649.0,63,2025-12-21 13:28:23.053953,,,,,,,,, +0.1351853907108307,0.0008610415970906615,0.1351853907108307,15699.0,63,2025-12-21 13:36:33.096298,,,,,,,,, +0.1520153433084488,0.0005169675569050014,0.1520153433084488,15749.0,63,2025-12-21 13:45:22.285287,,,,,,,,, +0.08950437605381012,0.0002997166884597391,0.08950437605381012,15799.0,63,2025-12-21 13:56:17.696628,,,,,,,,, +0.1777314394712448,0.0009593984577804804,0.1777314394712448,15849.0,63,2025-12-21 14:10:59.309465,,,,,,,,, +,,,,63,2025-12-21 14:19:42.323051,0.21742944419384003,0.002706316066905856,0.21742944419384003,0.19407358765602112,0.0016092726727947593,0.19407358765602112,,, +,,,,63,2025-12-21 14:19:54.126281,,,,,,,0.23660817742347717,0.0034728774335235357,0.23660817742347717 +0.0948498472571373,0.0003533830167725682,0.0948498472571373,15899.0,64,2025-12-21 14:26:11.970748,,,,,,,,, +0.14022067189216614,0.0006796946399845183,0.14022067189216614,15949.0,64,2025-12-21 14:34:52.464659,,,,,,,,, +0.22788585722446442,0.0008258463349193335,0.22788585722446442,15999.0,64,2025-12-21 14:43:04.384242,,,,,,,,, +0.12502500414848328,0.000684955099131912,0.12502500414848328,16049.0,64,2025-12-21 14:52:10.190008,,,,,,,,, +0.2009742259979248,0.0008355611353181303,0.2009742259979248,16099.0,64,2025-12-21 14:59:15.214998,,,,,,,,, +,,,,64,2025-12-21 15:06:27.801409,0.24076363444328308,0.003953567706048489,0.24076363444328308,0.1816411316394806,0.0012193307047709823,0.1816411316394806,,, +,,,,64,2025-12-21 15:06:39.986725,,,,,,,0.23387376964092255,0.0038859774358570576,0.23387376964092255 +0.07290055602788925,0.0002622588071972132,0.07290055602788925,16149.0,65,2025-12-21 15:14:52.629303,,,,,,,,, +0.21933138370513916,0.0012321777176111937,0.21933138370513916,16199.0,65,2025-12-21 15:23:07.080047,,,,,,,,, +0.23859497904777527,0.022998394444584846,0.23859497904777527,16249.0,65,2025-12-21 15:31:25.075231,,,,,,,,, +0.30505460500717163,0.0012821994023397565,0.30505460500717163,16299.0,65,2025-12-21 15:41:35.680897,,,,,,,,, +0.17483462393283844,0.0007510674186050892,0.17483462393283844,16349.0,65,2025-12-21 15:52:03.599878,,,,,,,,, +,,,,65,2025-12-21 16:03:19.762749,0.21158061921596527,0.0031909123063087463,0.21158061921596527,0.2166738659143448,0.0017268036026507616,0.2166738659143448,,, +,,,,65,2025-12-21 16:03:31.890722,,,,,,,0.23309126496315002,0.004310032818466425,0.23309126496315002 +0.1812419593334198,0.0006480177398771048,0.1812419593334198,16399.0,66,2025-12-21 16:10:39.484449,,,,,,,,, +0.3843080401420593,0.004733686801046133,0.3843080401420593,16449.0,66,2025-12-21 16:19:05.897909,,,,,,,,, +0.31946080923080444,0.0025996763724833727,0.31946080923080444,16499.0,66,2025-12-21 16:31:17.016226,,,,,,,,, +0.2672787308692932,0.0013003594940528274,0.2672787308692932,16549.0,66,2025-12-21 16:40:33.498224,,,,,,,,, +0.3398089110851288,0.0021498671267181635,0.3398089110851288,16599.0,66,2025-12-21 16:49:10.913230,,,,,,,,, +,,,,66,2025-12-21 16:59:05.947292,0.20940692722797394,0.0015501651214435697,0.20940692722797394,0.22746558487415314,0.003007701598107815,0.22746558487415314,,, +,,,,66,2025-12-21 16:59:17.970383,,,,,,,0.22762393951416016,0.0020908513106405735,0.22762393951416016 +0.18445514142513275,0.0006874430691823363,0.18445514142513275,16649.0,67,2025-12-21 17:07:20.789412,,,,,,,,, +0.16005519032478333,0.0006752509507350624,0.16005519032478333,16699.0,67,2025-12-21 17:21:59.564780,,,,,,,,, +0.30332544445991516,0.030364887788891792,0.30332544445991516,16749.0,67,2025-12-21 17:32:03.473761,,,,,,,,, +0.06395985186100006,0.00023633439559489489,0.06395985186100006,16799.0,67,2025-12-21 17:39:23.974020,,,,,,,,, +0.1599525362253189,0.0008697285666130483,0.1599525362253189,16849.0,67,2025-12-21 17:50:14.062635,,,,,,,,, +,,,,67,2025-12-21 17:57:17.920207,0.2597620189189911,0.002242207759991288,0.2597620189189911,0.24502147734165192,0.0021368819288909435,0.24502147734165192,,, +,,,,67,2025-12-21 17:57:29.721595,,,,,,,0.23411378264427185,0.003671460784971714,0.23411378264427185 +0.29485198855400085,0.001616707886569202,0.29485198855400085,16899.0,68,2025-12-21 18:07:07.120694,,,,,,,,, +0.16113488376140594,0.0005584213067777455,0.16113488376140594,16949.0,68,2025-12-21 18:15:18.348443,,,,,,,,, +0.2805078625679016,0.0060128807090222836,0.2805078625679016,16999.0,68,2025-12-21 18:24:55.937451,,,,,,,,, +0.22988712787628174,0.0009359440300613642,0.22988712787628174,17049.0,68,2025-12-21 18:33:58.721373,,,,,,,,, +0.18644404411315918,0.0009163842769339681,0.18644404411315918,17099.0,68,2025-12-21 18:44:17.417460,,,,,,,,, +,,,,68,2025-12-21 18:50:23.439025,0.2373511791229248,0.0015958183212205768,0.2373511791229248,0.20648200809955597,0.005095895379781723,0.20648200809955597,,, +,,,,68,2025-12-21 18:50:35.535311,,,,,,,0.21226991713047028,0.0023720383178442717,0.21226991713047028 +0.09862842410802841,0.0003613290027715266,0.09862842410802841,17149.0,69,2025-12-21 18:58:09.728590,,,,,,,,, +0.21720625460147858,0.000973215966951102,0.21720625460147858,17199.0,69,2025-12-21 19:10:52.147368,,,,,,,,, +0.2208031564950943,0.0008869164739735425,0.2208031564950943,17249.0,69,2025-12-21 19:24:45.328960,,,,,,,,, +0.1188981831073761,0.00040999066550284624,0.1188981831073761,17299.0,69,2025-12-21 19:39:22.712659,,,,,,,,, +0.28558939695358276,0.0015539333689957857,0.28558939695358276,17349.0,69,2025-12-21 19:45:39.074821,,,,,,,,, +,,,,69,2025-12-21 19:53:14.030674,0.185877725481987,0.002375610638409853,0.185877725481987,0.2250719517469406,0.0016733434749767184,0.2250719517469406,,, +,,,,69,2025-12-21 19:53:26.222349,,,,,,,0.21760490536689758,0.0032680798321962357,0.21760490536689758 +0.27118173241615295,0.011475908569991589,0.27118173241615295,17399.0,70,2025-12-21 20:01:50.813710,,,,,,,,, +0.5475612282752991,0.014465072192251682,0.5475612282752991,17449.0,70,2025-12-21 20:13:37.376776,,,,,,,,, +0.14529168605804443,0.0006460603326559067,0.14529168605804443,17499.0,70,2025-12-21 20:23:09.988562,,,,,,,,, +0.26586049795150757,0.0012740748934447765,0.26586049795150757,17549.0,70,2025-12-21 20:35:12.806095,,,,,,,,, +0.2648029029369354,0.0013440604088827968,0.2648029029369354,17599.0,70,2025-12-21 20:48:11.283183,,,,,,,,, +,,,,70,2025-12-21 20:53:42.997210,0.27107423543930054,0.01503948774188757,0.27107423543930054,0.20926719903945923,0.0031252114567905664,0.20926719903945923,,, +,,,,70,2025-12-21 20:53:54.519006,,,,,,,0.24032922089099884,0.005712519399821758,0.24032922089099884 +0.1988135576248169,0.0030103521421551704,0.1988135576248169,17649.0,71,2025-12-21 21:03:55.728678,,,,,,,,, +0.10189615190029144,0.0003408128395676613,0.10189615190029144,17699.0,71,2025-12-21 21:23:54.324558,,,,,,,,, +0.20015312731266022,0.001142005086876452,0.20015312731266022,17749.0,71,2025-12-21 21:34:42.346686,,,,,,,,, +0.14353615045547485,0.000529570272192359,0.14353615045547485,17799.0,71,2025-12-21 21:44:00.625688,,,,,,,,, +0.17871028184890747,0.0008336319006048143,0.17871028184890747,17849.0,71,2025-12-21 21:52:55.474952,,,,,,,,, +,,,,71,2025-12-21 21:58:29.491967,0.19810768961906433,0.002979996846988797,0.19810768961906433,0.24404704570770264,0.0120169036090374,0.24404704570770264,,, +,,,,71,2025-12-21 21:58:41.332096,,,,,,,0.22890806198120117,0.0032407124526798725,0.22890806198120117 +0.33310991525650024,0.002335799392312765,0.33310991525650024,17899.0,72,2025-12-21 22:08:31.988508,,,,,,,,, +0.22614945471286774,0.0009382183197885752,0.22614945471286774,17949.0,72,2025-12-21 22:16:27.431884,,,,,,,,, +0.26258060336112976,0.007643965072929859,0.26258060336112976,17999.0,72,2025-12-21 22:23:39.525192,,,,,,,,, +0.4863688051700592,0.002856888808310032,0.4863688051700592,18049.0,72,2025-12-21 22:38:33.134988,,,,,,,,, +0.07870606333017349,0.0002724801015574485,0.07870606333017349,18099.0,72,2025-12-21 22:48:20.918066,,,,,,,,, +,,,,72,2025-12-21 22:55:31.837292,0.2237144410610199,0.003064165823161602,0.2237144410610199,0.22439803183078766,0.010308990254998207,0.22439803183078766,,, +,,,,72,2025-12-21 22:55:44.065343,,,,,,,0.22878743708133698,0.0032989627216011286,0.22878743708133698 +0.22948697209358215,0.0008874883060343564,0.22948697209358215,18149.0,73,2025-12-21 23:05:54.674535,,,,,,,,, +0.25101548433303833,0.0011161683360114694,0.25101548433303833,18199.0,73,2025-12-21 23:16:32.940515,,,,,,,,, +0.17241322994232178,0.00062671082559973,0.17241322994232178,18249.0,73,2025-12-21 23:30:00.046692,,,,,,,,, +0.16719301044940948,0.0005953976069577038,0.16719301044940948,18299.0,73,2025-12-21 23:45:14.169959,,,,,,,,, +0.2026631087064743,0.001158634084276855,0.2026631087064743,18349.0,73,2025-12-21 23:53:54.386968,,,,,,,,, +,,,,73,2025-12-21 23:59:41.492621,0.21334180235862732,0.0034810900688171387,0.21334180235862732,0.2069714069366455,0.002471707295626402,0.2069714069366455,,, +,,,,73,2025-12-21 23:59:53.838548,,,,,,,0.22472955286502838,0.004670469090342522,0.22472955286502838 +0.41990384459495544,0.002085707848891616,0.41990384459495544,18399.0,74,2025-12-22 00:15:00.183164,,,,,,,,, +0.26978006958961487,0.001819559489376843,0.26978006958961487,18449.0,74,2025-12-22 00:28:11.646971,,,,,,,,, +0.2097204625606537,0.0013353077229112387,0.2097204625606537,18499.0,74,2025-12-22 00:38:13.455316,,,,,,,,, +0.2878044545650482,0.00589582696557045,0.2878044545650482,18549.0,74,2025-12-22 00:47:45.180842,,,,,,,,, +0.10426462441682816,0.0006312242476269603,0.10426462441682816,18599.0,74,2025-12-22 00:57:28.768651,,,,,,,,, +,,,,74,2025-12-22 01:02:21.942680,0.23261123895645142,0.0038557113148272038,0.23261123895645142,0.2401410937309265,0.0024685123935341835,0.2401410937309265,,, +,,,,74,2025-12-22 01:02:34.340707,,,,,,,0.22202613949775696,0.004953788127750158,0.22202613949775696 +0.22146610915660858,0.0007495007012039423,0.22146610915660858,18649.0,75,2025-12-22 01:09:53.402935,,,,,,,,, +0.17437028884887695,0.02503213845193386,0.17437028884887695,18699.0,75,2025-12-22 01:19:52.117420,,,,,,,,, +0.34508851170539856,0.0014905125135555863,0.34508851170539856,18749.0,75,2025-12-22 01:31:50.063239,,,,,,,,, +0.12562261521816254,0.0004539481014944613,0.12562261521816254,18799.0,75,2025-12-22 01:40:14.249975,,,,,,,,, +,,,,75,2025-12-22 01:56:44.590556,0.22565573453903198,0.002288918709382415,0.22565573453903198,0.22588562965393066,0.0014955423539504409,0.22588562965393066,,, +,,,,75,2025-12-22 01:56:56.597542,,,,,,,0.21846595406532288,0.003653167746961117,0.21846595406532288 +0.20736418664455414,0.000783176685217768,0.20736418664455414,18849.0,76,2025-12-22 01:59:41.222508,,,,,,,,, +0.1609521508216858,0.0008746955427341163,0.1609521508216858,18899.0,76,2025-12-22 02:07:55.021535,,,,,,,,, +0.10283908993005753,0.00035161044797860086,0.10283908993005753,18949.0,76,2025-12-22 02:19:46.006911,,,,,,,,, +0.3366435468196869,0.0017527226591482759,0.3366435468196869,18999.0,76,2025-12-22 02:28:58.860026,,,,,,,,, +0.16251489520072937,0.0005702587659470737,0.16251489520072937,19049.0,76,2025-12-22 02:40:00.823787,,,,,,,,, +,,,,76,2025-12-22 02:56:29.820960,0.24299602210521698,0.00316228112205863,0.24299602210521698,0.22055584192276,0.005489077884703875,0.22055584192276,,, +,,,,76,2025-12-22 02:56:42.139764,,,,,,,0.232741117477417,0.003289698390290141,0.232741117477417 +0.23057836294174194,0.0009078044095076621,0.23057836294174194,19099.0,77,2025-12-22 02:59:55.401107,,,,,,,,, +0.07814810425043106,0.0003729503077920526,0.07814810425043106,19149.0,77,2025-12-22 03:09:14.054384,,,,,,,,, +0.07426266372203827,0.0002530798374209553,0.07426266372203827,19199.0,77,2025-12-22 03:18:46.923902,,,,,,,,, +0.17891158163547516,0.0006836744723841548,0.17891158163547516,19249.0,77,2025-12-22 03:27:05.487957,,,,,,,,, +0.330744206905365,0.004906383343040943,0.330744206905365,19299.0,77,2025-12-22 03:36:52.890276,,,,,,,,, +,,,,77,2025-12-22 03:48:51.813829,0.1777004450559616,0.0014399670762941241,0.1777004450559616,0.198439821600914,0.0026896565686911345,0.198439821600914,,, +,,,,77,2025-12-22 03:49:04.339152,,,,,,,0.238174170255661,0.004042177461087704,0.238174170255661 +0.26706162095069885,0.001478002523072064,0.26706162095069885,19349.0,78,2025-12-22 03:51:11.864620,,,,,,,,, +0.3606244921684265,0.0027850100304931402,0.3606244921684265,19399.0,78,2025-12-22 04:03:58.141173,,,,,,,,, +0.25750282406806946,0.0012628018157556653,0.25750282406806946,19449.0,78,2025-12-22 04:12:57.610084,,,,,,,,, +0.3980163633823395,0.02470044605433941,0.3980163633823395,19499.0,78,2025-12-22 04:22:43.225416,,,,,,,,, +0.1284455806016922,0.0010446130763739347,0.1284455806016922,19549.0,78,2025-12-22 04:37:27.700023,,,,,,,,, +,,,,78,2025-12-22 04:48:27.989519,0.20127901434898376,0.002039450453594327,0.20127901434898376,0.20726710557937622,0.004359718877822161,0.20726710557937622,,, +,,,,78,2025-12-22 04:48:39.796903,,,,,,,0.22998321056365967,0.004722464829683304,0.22998321056365967 +0.33354923129081726,0.0021706733386963606,0.33354923129081726,19599.0,79,2025-12-22 04:51:42.846550,,,,,,,,, +0.1389978975057602,0.00047806152724660933,0.1389978975057602,19649.0,79,2025-12-22 05:02:55.341390,,,,,,,,, +0.17679736018180847,0.0014831373700872064,0.17679736018180847,19699.0,79,2025-12-22 05:14:55.221849,,,,,,,,, +0.25716838240623474,0.0027701142244040966,0.25716838240623474,19749.0,79,2025-12-22 05:30:18.909335,,,,,,,,, +0.2902907133102417,0.0038660108111798763,0.2902907133102417,19799.0,79,2025-12-22 05:43:19.508575,,,,,,,,, +,,,,79,2025-12-22 05:56:09.823344,0.23064033687114716,0.0022624654229730368,0.23064033687114716,0.22298663854599,0.003995291423052549,0.22298663854599,,, +,,,,79,2025-12-22 05:56:21.733948,,,,,,,0.2241797000169754,0.003790890332311392,0.2241797000169754 +0.3402614891529083,0.001517547876574099,0.3402614891529083,19849.0,80,2025-12-22 06:01:55.062463,,,,,,,,, +0.48404285311698914,0.0040558320470154285,0.48404285311698914,19899.0,80,2025-12-22 06:08:27.960728,,,,,,,,, +0.3730262219905853,0.11931353807449341,0.3730262219905853,19949.0,80,2025-12-22 06:20:41.675334,,,,,,,,, +0.28758037090301514,0.0016916252207010984,0.28758037090301514,19999.0,80,2025-12-22 06:33:51.005134,,,,,,,,, +0.14267653226852417,0.0005116938846185803,0.14267653226852417,20049.0,80,2025-12-22 06:47:50.170147,,,,,,,,, +,,,,80,2025-12-22 06:59:00.281166,0.20275984704494476,0.002322825836017728,0.20275984704494476,0.18424580991268158,0.001694501843303442,0.18424580991268158,,, +,,,,80,2025-12-22 06:59:11.999274,,,,,,,0.22674699127674103,0.0033472352661192417,0.22674699127674103 +0.39359787106513977,0.002242381451651454,0.39359787106513977,20099.0,81,2025-12-22 07:04:08.490483,,,,,,,,, +0.19661693274974823,0.002497578039765358,0.19661693274974823,20149.0,81,2025-12-22 07:15:18.737935,,,,,,,,, +0.1165948361158371,0.0016509591368958354,0.1165948361158371,20199.0,81,2025-12-22 07:28:43.646404,,,,,,,,, +0.13211074471473694,0.0004530246078502387,0.13211074471473694,20249.0,81,2025-12-22 07:36:23.466701,,,,,,,,, +0.19111162424087524,0.0010193211492151022,0.19111162424087524,20299.0,81,2025-12-22 07:46:46.656060,,,,,,,,, +,,,,81,2025-12-22 07:57:59.590414,0.268335223197937,0.003429373959079385,0.268335223197937,0.21053078770637512,0.003843418788164854,0.21053078770637512,,, +,,,,81,2025-12-22 07:58:11.357558,,,,,,,0.22008049488067627,0.0022411325480788946,0.22008049488067627 +0.20476987957954407,0.0008057112572714686,0.20476987957954407,20349.0,82,2025-12-22 08:02:13.597027,,,,,,,,, +0.17094385623931885,0.0005764884990639985,0.17094385623931885,20399.0,82,2025-12-22 08:09:51.797465,,,,,,,,, +0.6086221933364868,0.15362951159477234,0.6086221933364868,20449.0,82,2025-12-22 08:18:11.774838,,,,,,,,, +0.16449452936649323,0.0006440142169594765,0.16449452936649323,20499.0,82,2025-12-22 08:26:28.762937,,,,,,,,, +0.022860916331410408,9.900396253215149e-05,0.022860916331410408,20549.0,82,2025-12-22 08:36:03.576385,,,,,,,,, +,,,,82,2025-12-22 08:51:58.853930,0.2128886729478836,0.0023691162932664156,0.2128886729478836,0.23172415792942047,0.005488288123160601,0.23172415792942047,,, +,,,,82,2025-12-22 08:52:10.638947,,,,,,,0.22307446599006653,0.003200220176950097,0.22307446599006653 +0.21954044699668884,0.004127454478293657,0.21954044699668884,20599.0,83,2025-12-22 08:55:13.645346,,,,,,,,, +0.3817281723022461,0.0035450770519673824,0.3817281723022461,20649.0,83,2025-12-22 09:01:47.893922,,,,,,,,, +0.26778534054756165,0.0016734699020162225,0.26778534054756165,20699.0,83,2025-12-22 09:11:08.217550,,,,,,,,, +0.13928845524787903,0.00046310070320032537,0.13928845524787903,20749.0,83,2025-12-22 09:18:03.714059,,,,,,,,, +0.19210247695446014,0.0008048872114159167,0.19210247695446014,20799.0,83,2025-12-22 09:30:34.862066,,,,,,,,, +,,,,83,2025-12-22 09:40:16.287302,0.22629617154598236,0.008458679541945457,0.22629617154598236,0.20452402532100677,0.0017422535456717014,0.20452402532100677,,, +,,,,83,2025-12-22 09:40:28.653869,,,,,,,0.22314058244228363,0.003379007102921605,0.22314058244228363 +0.34979650378227234,0.002211273880675435,0.34979650378227234,20849.0,84,2025-12-22 09:50:21.834957,,,,,,,,, +0.237874373793602,0.0019187472062185407,0.237874373793602,20899.0,84,2025-12-22 09:58:08.461952,,,,,,,,, +0.09977726638317108,0.0003718852822203189,0.09977726638317108,20949.0,84,2025-12-22 10:05:15.294753,,,,,,,,, +0.17386668920516968,0.0006387098110280931,0.17386668920516968,20999.0,84,2025-12-22 10:15:29.407250,,,,,,,,, +0.36968547105789185,0.0046910387463867664,0.36968547105789185,21049.0,84,2025-12-22 10:27:23.237307,,,,,,,,, +,,,,84,2025-12-22 10:37:14.501874,0.23495110869407654,0.0024755948688834906,0.23495110869407654,0.2157198190689087,0.009291145950555801,0.2157198190689087,,, +,,,,84,2025-12-22 10:37:26.317245,,,,,,,0.23998981714248657,0.005277659278362989,0.23998981714248657 +0.2942076325416565,0.03968854248523712,0.2942076325416565,21099.0,85,2025-12-22 10:42:04.530736,,,,,,,,, +0.5013375282287598,0.025571629405021667,0.5013375282287598,21149.0,85,2025-12-22 10:52:53.127219,,,,,,,,, +0.308855801820755,0.0014920857502147555,0.308855801820755,21199.0,85,2025-12-22 11:07:50.123160,,,,,,,,, +0.20215071737766266,0.0006987590459175408,0.20215071737766266,21249.0,85,2025-12-22 11:14:40.346206,,,,,,,,, +0.22938503324985504,0.0014591605868190527,0.22938503324985504,21299.0,85,2025-12-22 11:24:44.059033,,,,,,,,, +,,,,85,2025-12-22 11:34:36.792288,0.22696876525878906,0.001597434631548822,0.22696876525878906,0.190904438495636,0.010691613890230656,0.190904438495636,,, +,,,,85,2025-12-22 11:34:49.155008,,,,,,,0.21785584092140198,0.00310503295622766,0.21785584092140198 +0.265210896730423,0.02977568656206131,0.265210896730423,21349.0,86,2025-12-22 11:40:51.794403,,,,,,,,, +0.20590342581272125,0.002024566289037466,0.20590342581272125,21399.0,86,2025-12-22 11:52:03.413840,,,,,,,,, +0.3982815444469452,0.0031579588539898396,0.3982815444469452,21449.0,86,2025-12-22 12:02:16.940947,,,,,,,,, +0.11810692399740219,0.0008473900379613042,0.11810692399740219,21499.0,86,2025-12-22 12:09:59.139033,,,,,,,,, +0.3485708236694336,0.004244843497872353,0.3485708236694336,21549.0,86,2025-12-22 12:21:27.821152,,,,,,,,, +,,,,86,2025-12-22 12:32:33.010225,0.2316179722547531,0.0020209550857543945,0.2316179722547531,0.2312907725572586,0.00452403211966157,0.2312907725572586,,, +,,,,86,2025-12-22 12:32:45.560059,,,,,,,0.22835101187229156,0.004302719607949257,0.22835101187229156 +0.09213429689407349,0.00036689749686047435,0.09213429689407349,21599.0,87,2025-12-22 12:40:46.136172,,,,,,,,, +0.4477333128452301,0.009600416757166386,0.4477333128452301,21649.0,87,2025-12-22 12:52:53.488057,,,,,,,,, +0.19776147603988647,0.0007827047957107425,0.19776147603988647,21699.0,87,2025-12-22 13:03:27.717480,,,,,,,,, +0.059047866612672806,0.00021293495956342667,0.059047866612672806,21749.0,87,2025-12-22 13:12:19.911490,,,,,,,,, +0.42553195357322693,0.023128731176257133,0.42553195357322693,21799.0,87,2025-12-22 13:23:37.668518,,,,,,,,, +,,,,87,2025-12-22 13:33:38.024715,0.20735344290733337,0.0015542696928605437,0.20735344290733337,0.22695620357990265,0.00400313176214695,0.22695620357990265,,, +,,,,87,2025-12-22 13:33:50.223361,,,,,,,0.21282513439655304,0.003397040069103241,0.21282513439655304 +0.26410040259361267,0.0017196888802573085,0.26410040259361267,21849.0,88,2025-12-22 13:39:15.053580,,,,,,,,, +0.12557287514209747,0.0004423458594828844,0.12557287514209747,21899.0,88,2025-12-22 13:46:58.964695,,,,,,,,, +0.17627201974391937,0.0007710015634074807,0.17627201974391937,21949.0,88,2025-12-22 13:59:02.085303,,,,,,,,, +0.295198917388916,0.0018264498794451356,0.295198917388916,21999.0,88,2025-12-22 14:06:38.121080,,,,,,,,, +0.127503901720047,0.0004308718489482999,0.127503901720047,22049.0,88,2025-12-22 14:15:22.464969,,,,,,,,, +,,,,88,2025-12-22 14:31:27.729526,0.23451493680477142,0.002442962024360895,0.23451493680477142,0.19384083151817322,0.001481217099353671,0.19384083151817322,,, +,,,,88,2025-12-22 14:31:40.089907,,,,,,,0.22477133572101593,0.003923539537936449,0.22477133572101593 +0.2942344546318054,0.0022135861217975616,0.2942344546318054,22099.0,89,2025-12-22 14:40:45.359751,,,,,,,,, +0.13950440287590027,0.0004648206813726574,0.13950440287590027,22149.0,89,2025-12-22 14:48:50.606456,,,,,,,,, +0.07813232392072678,0.0002912362397182733,0.07813232392072678,22199.0,89,2025-12-22 14:58:20.577446,,,,,,,,, +0.09842760860919952,0.00033382390392944217,0.09842760860919952,22249.0,89,2025-12-22 15:07:13.415888,,,,,,,,, +0.16488946974277496,0.0005874242051504552,0.16488946974277496,22299.0,89,2025-12-22 15:16:37.794066,,,,,,,,, +,,,,89,2025-12-22 15:25:05.693113,0.22905540466308594,0.0032269556540995836,0.22905540466308594,0.2564850449562073,0.009152314625680447,0.2564850449562073,,, +,,,,89,2025-12-22 15:25:17.644949,,,,,,,0.2163974493741989,0.00322523876093328,0.2163974493741989 +0.2771212160587311,0.001491760485805571,0.2771212160587311,22349.0,90,2025-12-22 15:34:33.769537,,,,,,,,, +0.15513554215431213,0.0005761882057413459,0.15513554215431213,22399.0,90,2025-12-22 15:42:17.543939,,,,,,,,, +0.16500817239284515,0.0006722951075062156,0.16500817239284515,22449.0,90,2025-12-22 15:52:46.722167,,,,,,,,, +0.3133590519428253,0.0056064547970891,0.3133590519428253,22499.0,90,2025-12-22 16:03:07.366440,,,,,,,,, +0.3522203266620636,0.010427051223814487,0.3522203266620636,22549.0,90,2025-12-22 16:13:26.876523,,,,,,,,, +,,,,90,2025-12-22 16:22:14.715168,0.18732233345508575,0.0019527884433045983,0.18732233345508575,0.21709051728248596,0.001474839518778026,0.21709051728248596,,, +,,,,90,2025-12-22 16:22:27.022594,,,,,,,0.22291769087314606,0.002839577617123723,0.22291769087314606 +0.19899408519268036,0.0010467765387147665,0.19899408519268036,22599.0,91,2025-12-22 16:28:37.811750,,,,,,,,, +0.08258160203695297,0.0002868863521143794,0.08258160203695297,22649.0,91,2025-12-22 16:37:22.544654,,,,,,,,, +0.38032469153404236,0.010655351914465427,0.38032469153404236,22699.0,91,2025-12-22 16:47:46.280156,,,,,,,,, +0.06999126076698303,0.000247322692302987,0.06999126076698303,22749.0,91,2025-12-22 17:05:52.925664,,,,,,,,, +0.45785561203956604,0.04112642630934715,0.45785561203956604,22799.0,91,2025-12-22 17:15:23.109609,,,,,,,,, +,,,,91,2025-12-22 17:22:09.755963,0.2483869194984436,0.0035428875125944614,0.2483869194984436,0.2307356894016266,0.003402512986212969,0.2307356894016266,,, +,,,,91,2025-12-22 17:22:22.107623,,,,,,,0.2318686991930008,0.003068904159590602,0.2318686991930008 +0.08249956369400024,0.00028165921685285866,0.08249956369400024,22849.0,92,2025-12-22 17:29:57.638417,,,,,,,,, +0.11120587587356567,0.0003777559904847294,0.11120587587356567,22899.0,92,2025-12-22 17:40:03.848775,,,,,,,,, +0.20943094789981842,0.0010397438891232014,0.20943094789981842,22949.0,92,2025-12-22 17:48:21.888221,,,,,,,,, +0.15089784562587738,0.0005521748098544776,0.15089784562587738,22999.0,92,2025-12-22 17:58:57.482320,,,,,,,,, +0.1840510368347168,0.0007846647058613598,0.1840510368347168,23049.0,92,2025-12-22 18:06:59.881955,,,,,,,,, +,,,,92,2025-12-22 18:14:13.574755,0.18756833672523499,0.0012874483363702893,0.18756833672523499,0.17472560703754425,0.001127934199757874,0.17472560703754425,,, +,,,,92,2025-12-22 18:14:36.409737,,,,,,,0.22062960267066956,0.002933803480118513,0.22062960267066956 +0.18608330190181732,0.0007789979572407901,0.18608330190181732,23099.0,93,2025-12-22 18:23:07.778571,,,,,,,,, +0.24417607486248016,0.002070655580610037,0.24417607486248016,23149.0,93,2025-12-22 18:31:31.647822,,,,,,,,, +0.3936619758605957,0.002430011983960867,0.3936619758605957,23199.0,93,2025-12-22 18:46:47.067759,,,,,,,,, +0.24249829351902008,0.006780387368053198,0.24249829351902008,23249.0,93,2025-12-22 18:59:50.440405,,,,,,,,, +0.16688884794712067,0.0006080708117224276,0.16688884794712067,23299.0,93,2025-12-22 19:09:31.504648,,,,,,,,, +,,,,93,2025-12-22 19:16:55.631770,0.20363305509090424,0.0028538622427731752,0.20363305509090424,0.2369261533021927,0.0028248950839042664,0.2369261533021927,,, +,,,,93,2025-12-22 19:17:07.882618,,,,,,,0.21783547103405,0.0038799946196377277,0.21783547103405 +0.0754120871424675,0.0002747898979578167,0.0754120871424675,23349.0,94,2025-12-22 19:25:07.180863,,,,,,,,, +0.2968686521053314,0.006777447182685137,0.2968686521053314,23399.0,94,2025-12-22 19:36:20.724249,,,,,,,,, +0.5019396543502808,0.004143447615206242,0.5019396543502808,23449.0,94,2025-12-22 19:45:07.313789,,,,,,,,, +0.26960599422454834,0.0009267522837035358,0.26960599422454834,23499.0,94,2025-12-22 19:53:11.696721,,,,,,,,, +0.19682668149471283,0.000781475508119911,0.19682668149471283,23549.0,94,2025-12-22 20:01:58.159259,,,,,,,,, +,,,,94,2025-12-22 20:07:53.802014,0.21199779212474823,0.0015695927431806922,0.21199779212474823,0.1817837357521057,0.0008735775481909513,0.1817837357521057,,, +,,,,94,2025-12-22 20:08:06.106762,,,,,,,0.23260623216629028,0.003978390246629715,0.23260623216629028 +0.23173943161964417,0.0009780890541151166,0.23173943161964417,23599.0,95,2025-12-22 20:15:40.469485,,,,,,,,, +0.3504475951194763,0.002356224460527301,0.3504475951194763,23649.0,95,2025-12-22 20:24:27.395526,,,,,,,,, +0.1478186845779419,0.0005363677628338337,0.1478186845779419,23699.0,95,2025-12-22 20:40:12.802478,,,,,,,,, +0.35721060633659363,0.008737360127270222,0.35721060633659363,23749.0,95,2025-12-22 20:47:40.791191,,,,,,,,, +0.35713449120521545,0.0020153408404439688,0.35713449120521545,23799.0,95,2025-12-22 20:55:11.068367,,,,,,,,, +,,,,95,2025-12-22 21:00:59.671981,0.1855478584766388,0.00650804815813899,0.1855478584766388,0.15854792296886444,0.0009782365523278713,0.15854792296886444,,, +,,,,95,2025-12-22 21:01:21.086071,,,,,,,0.23046202957630157,0.0038017425686120987,0.23046202957630157 +0.15257905423641205,0.0005681777838617563,0.15257905423641205,23849.0,96,2025-12-22 21:10:02.485758,,,,,,,,, +0.2950991690158844,0.0017614559037610888,0.2950991690158844,23899.0,96,2025-12-22 21:21:37.994271,,,,,,,,, +0.29994937777519226,0.0013828834053128958,0.29994937777519226,23949.0,96,2025-12-22 21:31:36.823654,,,,,,,,, +0.2443811148405075,0.0012833791552111506,0.2443811148405075,23999.0,96,2025-12-22 21:39:15.696236,,,,,,,,, +0.1539241224527359,0.0007916343747638166,0.1539241224527359,24049.0,96,2025-12-22 21:51:16.361057,,,,,,,,, +,,,,96,2025-12-22 21:56:37.700582,0.2178180068731308,0.0035879118368029594,0.2178180068731308,0.2045058012008667,0.006620465312153101,0.2045058012008667,,, +,,,,96,2025-12-22 21:56:49.908962,,,,,,,0.2286844700574875,0.003338215406984091,0.2286844700574875 +0.1849365532398224,0.0012995477300137281,0.1849365532398224,24099.0,97,2025-12-22 22:07:11.016181,,,,,,,,, +0.08659453690052032,0.0003021917655132711,0.08659453690052032,24149.0,97,2025-12-22 22:15:45.618297,,,,,,,,, +0.2334304302930832,0.001014446490444243,0.2334304302930832,24199.0,97,2025-12-22 22:29:56.226133,,,,,,,,, +0.2678828537464142,0.0020134581718593836,0.2678828537464142,24249.0,97,2025-12-22 22:37:48.303064,,,,,,,,, +0.1880275160074234,0.0007630460895597935,0.1880275160074234,24299.0,97,2025-12-22 22:47:08.054905,,,,,,,,, +,,,,97,2025-12-22 22:52:35.312073,0.2710055410861969,0.006315015256404877,0.2710055410861969,0.22013676166534424,0.0028331491630524397,0.22013676166534424,,, +,,,,97,2025-12-22 22:52:46.952213,,,,,,,0.23374706506729126,0.0033778524957597256,0.23374706506729126 +0.3734835088253021,0.0020439026411622763,0.3734835088253021,24349.0,98,2025-12-22 23:07:47.068398,,,,,,,,, +0.16184067726135254,0.0007179996464401484,0.16184067726135254,24399.0,98,2025-12-22 23:19:43.097016,,,,,,,,, +0.2598826587200165,0.001829740358516574,0.2598826587200165,24449.0,98,2025-12-22 23:30:29.993146,,,,,,,,, +0.38606691360473633,0.0020767718087881804,0.38606691360473633,24499.0,98,2025-12-22 23:37:33.699926,,,,,,,,, +0.19636717438697815,0.002027550945058465,0.19636717438697815,24549.0,98,2025-12-22 23:44:54.819182,,,,,,,,, +,,,,98,2025-12-22 23:50:02.139681,0.1871945858001709,0.0022472022101283073,0.1871945858001709,0.2590456008911133,0.006126639433205128,0.2590456008911133,,, +,,,,98,2025-12-22 23:50:13.676342,,,,,,,0.2142275869846344,0.0036943310406059027,0.2142275869846344 +0.24721606075763702,0.001009974628686905,0.24721606075763702,24599.0,99,2025-12-23 00:01:14.000871,,,,,,,,, +0.09604233503341675,0.00032940812525339425,0.09604233503341675,24649.0,99,2025-12-23 00:14:46.194045,,,,,,,,, +0.07548261433839798,0.0002646180219016969,0.07548261433839798,24699.0,99,2025-12-23 00:23:40.721876,,,,,,,,, +0.12438985705375671,0.0005223738844506443,0.12438985705375671,24749.0,99,2025-12-23 00:30:28.141790,,,,,,,,, +0.235483780503273,0.0008893791236914694,0.235483780503273,24799.0,99,2025-12-23 00:39:22.026439,,,,,,,,, +,,,,99,2025-12-23 00:44:17.806559,0.22060438990592957,0.006920576561242342,0.22060438990592957,0.24170784652233124,0.002367684617638588,0.24170784652233124,,, +,,,,99,2025-12-23 00:44:30.634521,,,,,,,0.22243867814540863,0.003476253943517804,0.22243867814540863 +0.3132297098636627,0.0013252765638753772,0.3132297098636627,24849.0,100,2025-12-23 00:56:52.168396,,,,,,,,, +0.10424655675888062,0.00035450278664939106,0.10424655675888062,24899.0,100,2025-12-23 01:06:05.167563,,,,,,,,, +0.12801574170589447,0.0011511418269947171,0.12801574170589447,24949.0,100,2025-12-23 01:23:18.228891,,,,,,,,, +0.3246723711490631,0.005427233874797821,0.3246723711490631,24999.0,100,2025-12-23 01:34:58.008297,,,,,,,,, +,,,,100,2025-12-23 01:49:07.679492,0.23842324316501617,0.0024427997414022684,0.23842324316501617,0.19048936665058136,0.001155363628640771,0.19048936665058136,,, +,,,,100,2025-12-23 01:49:19.172454,,,,,,,0.22831162810325623,0.004331402014940977,0.22831162810325623 +0.4272844195365906,0.0029894134495407343,0.4272844195365906,25049.0,101,2025-12-23 01:51:44.685372,,,,,,,,, +0.17901301383972168,0.0006192289292812347,0.17901301383972168,25099.0,101,2025-12-23 02:00:02.716837,,,,,,,,, +0.1869506537914276,0.009433153085410595,0.1869506537914276,25149.0,101,2025-12-23 02:07:27.018889,,,,,,,,, +0.16574735939502716,0.0006092864205129445,0.16574735939502716,25199.0,101,2025-12-23 02:16:59.629546,,,,,,,,, +0.4439762830734253,0.024262845516204834,0.4439762830734253,25249.0,101,2025-12-23 02:25:41.505427,,,,,,,,, +,,,,101,2025-12-23 02:42:28.601141,0.24905698001384735,0.004000536631792784,0.24905698001384735,0.253801554441452,0.0038065591361373663,0.253801554441452,,, +,,,,101,2025-12-23 02:42:40.886028,,,,,,,0.22963015735149384,0.004742517601698637,0.22963015735149384 +0.0885055810213089,0.00031812296947464347,0.0885055810213089,25299.0,102,2025-12-23 02:46:48.266523,,,,,,,,, +0.35308560729026794,0.12231937795877457,0.35308560729026794,25349.0,102,2025-12-23 02:58:08.917930,,,,,,,,, +0.43242448568344116,0.003321293042972684,0.43242448568344116,25399.0,102,2025-12-23 03:07:59.081815,,,,,,,,, +0.1082698255777359,0.0004859441251028329,0.1082698255777359,25449.0,102,2025-12-23 03:17:23.404221,,,,,,,,, +0.21332022547721863,0.0019307165639474988,0.21332022547721863,25499.0,102,2025-12-23 03:29:35.404160,,,,,,,,, +,,,,102,2025-12-23 03:43:05.741223,0.24375183880329132,0.013543839566409588,0.24375183880329132,0.19543524086475372,0.0054176016710698605,0.19543524086475372,,, +,,,,102,2025-12-23 03:43:17.680361,,,,,,,0.22001495957374573,0.00375683163292706,0.22001495957374573 +0.03564287722110748,0.000134814836201258,0.03564287722110748,25549.0,103,2025-12-23 03:45:04.676322,,,,,,,,, +0.3071652352809906,0.00300406850874424,0.3071652352809906,25599.0,103,2025-12-23 03:57:30.960180,,,,,,,,, +0.07753932476043701,0.00033344340045005083,0.07753932476043701,25649.0,103,2025-12-23 04:06:09.242347,,,,,,,,, +0.3006564676761627,0.0020109107717871666,0.3006564676761627,25699.0,103,2025-12-23 04:17:23.961309,,,,,,,,, +0.1088305190205574,0.0004548527067527175,0.1088305190205574,25749.0,103,2025-12-23 04:24:30.446614,,,,,,,,, +,,,,103,2025-12-23 04:43:39.528000,0.22329029440879822,0.00233942992053926,0.22329029440879822,0.21616941690444946,0.0030353348702192307,0.21616941690444946,,, +,,,,103,2025-12-23 04:43:51.434989,,,,,,,0.23029790818691254,0.004567583091557026,0.23029790818691254 +0.24707520008087158,0.003180644940584898,0.24707520008087158,25799.0,104,2025-12-23 04:47:25.294479,,,,,,,,, +0.2517910897731781,0.006103683263063431,0.2517910897731781,25849.0,104,2025-12-23 04:56:10.571409,,,,,,,,, +0.146937757730484,0.0005452324985526502,0.146937757730484,25899.0,104,2025-12-23 05:08:42.035643,,,,,,,,, +0.24496720731258392,0.0008765370585024357,0.24496720731258392,25949.0,104,2025-12-23 05:18:47.267541,,,,,,,,, +0.20016823709011078,0.0008040955872274935,0.20016823709011078,25999.0,104,2025-12-23 05:33:22.948668,,,,,,,,, +,,,,104,2025-12-23 05:44:23.236372,0.2170526534318924,0.002068069763481617,0.2170526534318924,0.21160191297531128,0.002063080668449402,0.21160191297531128,,, +,,,,104,2025-12-23 05:44:35.490750,,,,,,,0.2173515111207962,0.0035390611737966537,0.2173515111207962 +0.3235713839530945,0.0017047326546162367,0.3235713839530945,26049.0,105,2025-12-23 05:48:03.327019,,,,,,,,, +0.28611060976982117,0.00221177632920444,0.28611060976982117,26099.0,105,2025-12-23 05:59:48.797369,,,,,,,,, +0.11621294170618057,0.00043613649904727936,0.11621294170618057,26149.0,105,2025-12-23 06:12:10.142984,,,,,,,,, +0.34479039907455444,0.0038173093926161528,0.34479039907455444,26199.0,105,2025-12-23 06:20:55.114437,,,,,,,,, +0.2126975953578949,0.0010062602814286947,0.2126975953578949,26249.0,105,2025-12-23 06:29:47.426734,,,,,,,,, +,,,,105,2025-12-23 06:41:52.235621,0.16463913023471832,0.0011282899649813771,0.16463913023471832,0.2058623731136322,0.003697263542562723,0.2058623731136322,,, +,,,,105,2025-12-23 06:42:04.624620,,,,,,,0.23146605491638184,0.003056371584534645,0.23146605491638184 +0.21243086457252502,0.0007708342163823545,0.21243086457252502,26299.0,106,2025-12-23 06:46:00.140588,,,,,,,,, +0.25504595041275024,0.002674056915566325,0.25504595041275024,26349.0,106,2025-12-23 06:56:27.040967,,,,,,,,, +0.20770590007305145,0.0007851432892493904,0.20770590007305145,26399.0,106,2025-12-23 07:11:23.178104,,,,,,,,, +0.2019682228565216,0.0007554931216873229,0.2019682228565216,26449.0,106,2025-12-23 07:20:39.533930,,,,,,,,, +0.15515251457691193,0.000581440341193229,0.15515251457691193,26499.0,106,2025-12-23 07:29:03.671922,,,,,,,,, +,,,,106,2025-12-23 07:38:39.905297,0.18167752027511597,0.001089625176973641,0.18167752027511597,0.22193360328674316,0.004166679456830025,0.22193360328674316,,, +,,,,106,2025-12-23 07:38:51.785067,,,,,,,0.21844929456710815,0.0033084568567574024,0.21844929456710815 +0.3136126399040222,0.005065009463578463,0.3136126399040222,26549.0,107,2025-12-23 07:41:56.605887,,,,,,,,, diff --git a/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/tf/events.out.tfevents.1766086530.node-0.2036.0 b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/tf/events.out.tfevents.1766086530.node-0.2036.0 new file mode 100644 index 0000000000000000000000000000000000000000..accb51b995a0e61c15006cb99b423791613c2fcb --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/logs/full_ct_3d_with_body_mask2/testtube/version_0/tf/events.out.tfevents.1766086530.node-0.2036.0 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ac1fa574aca51f340924aef70aba1fbc56556f6d3fb174c67882e0f87bc6ecbf +size 205386 diff --git a/Abnormal-CT-Generation-MultiDisease/main.py b/Abnormal-CT-Generation-MultiDisease/main.py new file mode 100644 index 0000000000000000000000000000000000000000..8775f89b0b4a1f417e7f5c9855236c046aa0cb13 --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/main.py @@ -0,0 +1,774 @@ +import argparse, os, sys, datetime, glob, importlib, csv +import numpy as np +import time +import torch +import torchvision +import enum + +# 保存原始实现(可选,用于调试或回滚) +_enum_format_orig = enum.Enum.__format__ + +def _enum_format_value(self, format_spec: str) -> str: + # 如果指定了 format 规范,就交给 value 本身的格式化;否则直接返回 value + return format(self.value, format_spec) if format_spec else str(self.value) + +# 全局替换 Enum.__format__ +enum.Enum.__format__ = _enum_format_value + +import pytorch_lightning as pl + +from packaging import version +from omegaconf import OmegaConf +from torch.utils.data import random_split, DataLoader, Dataset, Subset +from functools import partial +from PIL import Image + +from pytorch_lightning import seed_everything +from pytorch_lightning.trainer import Trainer +from pytorch_lightning.callbacks import ModelCheckpoint, Callback, LearningRateMonitor +from pytorch_lightning.utilities.distributed import rank_zero_only +from pytorch_lightning.utilities import rank_zero_info + +from ldm.data.base import Txt2ImgIterableBaseDataset +from ldm.util import instantiate_from_config + + +def get_parser(**parser_kwargs): + def str2bool(v): + if isinstance(v, bool): + return v + if v.lower() in ("yes", "true", "t", "y", "1"): + return True + elif v.lower() in ("no", "false", "f", "n", "0"): + return False + else: + raise argparse.ArgumentTypeError("Boolean value expected.") + + parser = argparse.ArgumentParser(**parser_kwargs) + parser.add_argument( + "-n", + "--name", + type=str, + const=True, + default="", + nargs="?", + help="postfix for logdir", + ) + parser.add_argument( + "-r", + "--resume", + type=str, + const=True, + default="", + nargs="?", + help="resume from logdir or checkpoint in logdir", + ) + parser.add_argument( + "-b", + "--base", + nargs="*", + metavar="base_config.yaml", + help="paths to base configs. Loaded from left-to-right. " + "Parameters can be overwritten or added with command-line options of the form `--key value`.", + default=list(), + ) + parser.add_argument( + "-t", + "--train", + type=str2bool, + const=True, + default=False, + nargs="?", + help="train", + ) + parser.add_argument( + "--no-test", + type=str2bool, + const=True, + default=False, + nargs="?", + help="disable test", + ) + parser.add_argument( + "-p", + "--project", + help="name of new or path to existing project" + ) + parser.add_argument( + "-d", + "--debug", + type=str2bool, + nargs="?", + const=True, + default=False, + help="enable post-mortem debugging", + ) + parser.add_argument( + "-s", + "--seed", + type=int, + default=23, + help="seed for seed_everything", + ) + parser.add_argument( + "-f", + "--postfix", + type=str, + default="", + help="post-postfix for default name", + ) + parser.add_argument( + "-l", + "--logdir", + type=str, + default="logs", + help="directory for logging dat shit", + ) + parser.add_argument( + "--scale_lr", + type=str2bool, + nargs="?", + const=True, + default=True, + help="scale base-lr by ngpu * batch_size * n_accumulate", + ) + return parser + + +def nondefault_trainer_args(opt): + parser = argparse.ArgumentParser() + parser = Trainer.add_argparse_args(parser) + args = parser.parse_args([]) + return sorted(k for k in vars(args) if getattr(opt, k) != getattr(args, k)) + + +class WrappedDataset(Dataset): + """Wraps an arbitrary object with __len__ and __getitem__ into a pytorch dataset""" + + def __init__(self, dataset): + self.data = dataset + + def __len__(self): + return len(self.data) + + def __getitem__(self, idx): + return self.data[idx] + + +def worker_init_fn(_): + worker_info = torch.utils.data.get_worker_info() + + dataset = worker_info.dataset + worker_id = worker_info.id + + if isinstance(dataset, Txt2ImgIterableBaseDataset): + split_size = dataset.num_records // worker_info.num_workers + # reset num_records to the true number to retain reliable length information + dataset.sample_ids = dataset.valid_ids[worker_id * split_size:(worker_id + 1) * split_size] + current_id = np.random.choice(len(np.random.get_state()[1]), 1) + return np.random.seed(np.random.get_state()[1][current_id] + worker_id) + else: + return np.random.seed(np.random.get_state()[1][0] + worker_id) + + +class DataModuleFromConfig(pl.LightningDataModule): + def __init__(self, batch_size, train=None, validation=None, test=None, predict=None, + wrap=False, num_workers=None, shuffle_test_loader=False, use_worker_init_fn=False, + shuffle_val_dataloader=False): + super().__init__() + self.batch_size = batch_size + self.dataset_configs = dict() + self.num_workers = num_workers if num_workers is not None else batch_size * 2 + self.use_worker_init_fn = use_worker_init_fn + if train is not None: + self.dataset_configs["train"] = train + self.train_dataloader = self._train_dataloader + if validation is not None: + self.dataset_configs["validation"] = validation + self.val_dataloader = partial(self._val_dataloader, shuffle=shuffle_val_dataloader) + if test is not None: + self.dataset_configs["test"] = test + self.test_dataloader = partial(self._test_dataloader, shuffle=shuffle_test_loader) + if predict is not None: + self.dataset_configs["predict"] = predict + self.predict_dataloader = self._predict_dataloader + self.wrap = wrap + + def prepare_data(self): + for data_cfg in self.dataset_configs.values(): + instantiate_from_config(data_cfg) + + def setup(self, stage=None): + self.datasets = dict( + (k, instantiate_from_config(self.dataset_configs[k])) + for k in self.dataset_configs) + if self.wrap: + for k in self.datasets: + self.datasets[k] = WrappedDataset(self.datasets[k]) + + def _train_dataloader(self): + is_iterable_dataset = isinstance(self.datasets['train'], Txt2ImgIterableBaseDataset) + if is_iterable_dataset or self.use_worker_init_fn: + init_fn = worker_init_fn + else: + init_fn = None + return DataLoader(self.datasets["train"], batch_size=self.batch_size, + num_workers=self.num_workers, shuffle=False if is_iterable_dataset else True, + worker_init_fn=init_fn) + + def _val_dataloader(self, shuffle=False): + if isinstance(self.datasets['validation'], Txt2ImgIterableBaseDataset) or self.use_worker_init_fn: + init_fn = worker_init_fn + else: + init_fn = None + return DataLoader(self.datasets["validation"], + batch_size=self.batch_size, + num_workers=self.num_workers, + worker_init_fn=init_fn, + shuffle=shuffle) + + def _test_dataloader(self, shuffle=False): + is_iterable_dataset = isinstance(self.datasets['train'], Txt2ImgIterableBaseDataset) + if is_iterable_dataset or self.use_worker_init_fn: + init_fn = worker_init_fn + else: + init_fn = None + + # do not shuffle dataloader for iterable dataset + shuffle = shuffle and (not is_iterable_dataset) + + return DataLoader(self.datasets["test"], batch_size=self.batch_size, + num_workers=self.num_workers, worker_init_fn=init_fn, shuffle=shuffle) + + def _predict_dataloader(self, shuffle=False): + if isinstance(self.datasets['predict'], Txt2ImgIterableBaseDataset) or self.use_worker_init_fn: + init_fn = worker_init_fn + else: + init_fn = None + return DataLoader(self.datasets["predict"], batch_size=self.batch_size, + num_workers=self.num_workers, worker_init_fn=init_fn) + + +class SetupCallback(Callback): + def __init__(self, resume, now, logdir, ckptdir, cfgdir, config, lightning_config): + super().__init__() + self.resume = resume + self.now = now + self.logdir = logdir + self.ckptdir = ckptdir + self.cfgdir = cfgdir + self.config = config + self.lightning_config = lightning_config + + def on_keyboard_interrupt(self, trainer, pl_module): + if trainer.global_rank == 0: + print("Summoning checkpoint.") + ckpt_path = os.path.join(self.ckptdir, "last.ckpt") + trainer.save_checkpoint(ckpt_path) + + def on_pretrain_routine_start(self, trainer, pl_module): + if trainer.global_rank == 0: + # Create logdirs and save configs + os.makedirs(self.logdir, exist_ok=True) + os.makedirs(self.ckptdir, exist_ok=True) + os.makedirs(self.cfgdir, exist_ok=True) + + if "callbacks" in self.lightning_config: + if 'metrics_over_trainsteps_checkpoint' in self.lightning_config['callbacks']: + os.makedirs(os.path.join(self.ckptdir, 'trainstep_checkpoints'), exist_ok=True) + print("Project config") + print(OmegaConf.to_yaml(self.config)) + OmegaConf.save(self.config, + os.path.join(self.cfgdir, "{}-project.yaml".format(self.now))) + + print("Lightning config") + print(OmegaConf.to_yaml(self.lightning_config)) + OmegaConf.save(OmegaConf.create({"lightning": self.lightning_config}), + os.path.join(self.cfgdir, "{}-lightning.yaml".format(self.now))) + + else: + # ModelCheckpoint callback created log directory --- remove it + if not self.resume and os.path.exists(self.logdir): + dst, name = os.path.split(self.logdir) + dst = os.path.join(dst, "child_runs", name) + os.makedirs(os.path.split(dst)[0], exist_ok=True) + try: + os.rename(self.logdir, dst) + except FileNotFoundError: + pass + + +class ImageLogger(Callback): + def __init__(self, batch_frequency, max_images, clamp=True, increase_log_steps=True, + rescale=True, disabled=False, log_on_batch_idx=False, log_first_step=False, + log_images_kwargs=None): + super().__init__() + self.rescale = rescale + self.batch_freq = batch_frequency + self.max_images = max_images + self.logger_log_images = { + pl.loggers.TestTubeLogger: self._testtube, + } + self.log_steps = [2 ** n for n in range(int(np.log2(self.batch_freq)) + 1)] + if not increase_log_steps: + self.log_steps = [self.batch_freq] + self.clamp = clamp + self.disabled = disabled + self.log_on_batch_idx = log_on_batch_idx + self.log_images_kwargs = log_images_kwargs if log_images_kwargs else {} + self.log_first_step = log_first_step + + @rank_zero_only + def _testtube(self, pl_module, images, batch_idx, split): + for k in images: + grid = torchvision.utils.make_grid(images[k]) + grid = (grid + 1.0) / 2.0 # -1,1 -> 0,1; c,h,w + + tag = f"{split}/{k}" + pl_module.logger.experiment.add_image( + tag, grid, + global_step=pl_module.global_step) + + @rank_zero_only + def log_local(self, save_dir, split, images, + global_step, current_epoch, batch_idx): + root = os.path.join(save_dir, "images", split) + for k in images: + grid = torchvision.utils.make_grid(images[k], nrow=4) + if self.rescale: + grid = (grid + 1.0) / 2.0 # -1,1 -> 0,1; c,h,w + grid = grid.transpose(0, 1).transpose(1, 2).squeeze(-1) + grid = grid.numpy() + grid = (grid * 255).astype(np.uint8) + filename = "{}_gs-{:06}_e-{:06}_b-{:06}.png".format( + k, + global_step, + current_epoch, + batch_idx) + path = os.path.join(root, filename) + os.makedirs(os.path.split(path)[0], exist_ok=True) + Image.fromarray(grid).save(path) + + def log_img(self, pl_module, batch, batch_idx, split="train"): + check_idx = batch_idx if self.log_on_batch_idx else pl_module.global_step + if (self.check_frequency(check_idx) and # batch_idx % self.batch_freq == 0 + hasattr(pl_module, "log_images") and + callable(pl_module.log_images) and + self.max_images > 0): + logger = type(pl_module.logger) + + is_train = pl_module.training + if is_train: + pl_module.eval() + + with torch.no_grad(): + images = pl_module.log_images(batch, split=split, **self.log_images_kwargs) + + for k in images: + N = min(images[k].shape[0], self.max_images) + images[k] = images[k][:N] + if isinstance(images[k], torch.Tensor): + images[k] = images[k].detach().cpu() + if self.clamp: + images[k] = torch.clamp(images[k], -1., 1.) + + self.log_local(pl_module.logger.save_dir, split, images, + pl_module.global_step, pl_module.current_epoch, batch_idx) + + logger_log_images = self.logger_log_images.get(logger, lambda *args, **kwargs: None) + logger_log_images(pl_module, images, pl_module.global_step, split) + + if is_train: + pl_module.train() + + def check_frequency(self, check_idx): + if ((check_idx % self.batch_freq) == 0 or (check_idx in self.log_steps)) and ( + check_idx > 0 or self.log_first_step): + try: + self.log_steps.pop(0) + except IndexError as e: + print(e) + pass + return True + return False + + def on_train_batch_end(self, trainer, pl_module, outputs, batch, batch_idx, dataloader_idx): + if not self.disabled and (pl_module.global_step > 0 or self.log_first_step): + self.log_img(pl_module, batch, batch_idx, split="train") + + def on_validation_batch_end(self, trainer, pl_module, outputs, batch, batch_idx, dataloader_idx): + # if not self.disabled and pl_module.global_step > 0: + # self.log_img(pl_module, batch, batch_idx, split="val") + if hasattr(pl_module, 'calibrate_grad_norm'): + if (pl_module.calibrate_grad_norm and batch_idx % 25 == 0) and batch_idx > 0: + self.log_gradients(trainer, pl_module, batch_idx=batch_idx) + + +class CUDACallback(Callback): + # see https://github.com/SeanNaren/minGPT/blob/master/mingpt/callback.py + def on_train_epoch_start(self, trainer, pl_module): + # Reset the memory use counter + torch.cuda.reset_peak_memory_stats(trainer.root_gpu) + torch.cuda.synchronize(trainer.root_gpu) + self.start_time = time.time() + + def on_train_epoch_end(self, trainer, pl_module, outputs): + torch.cuda.synchronize(trainer.root_gpu) + max_memory = torch.cuda.max_memory_allocated(trainer.root_gpu) / 2 ** 20 + epoch_time = time.time() - self.start_time + + try: + max_memory = trainer.training_type_plugin.reduce(max_memory) + epoch_time = trainer.training_type_plugin.reduce(epoch_time) + + rank_zero_info(f"Average Epoch time: {epoch_time:.2f} seconds") + rank_zero_info(f"Average Peak memory {max_memory:.2f}MiB") + except AttributeError: + pass + + +if __name__ == "__main__": + # custom parser to specify config files, train, test and debug mode, + # postfix, resume. + # `--key value` arguments are interpreted as arguments to the trainer. + # `nested.key=value` arguments are interpreted as config parameters. + # configs are merged from left-to-right followed by command line parameters. + + # model: + # base_learning_rate: float + # target: path to lightning module + # params: + # key: value + # data: + # target: main.DataModuleFromConfig + # params: + # batch_size: int + # wrap: bool + # train: + # target: path to train dataset + # params: + # key: value + # validation: + # target: path to validation dataset + # params: + # key: value + # test: + # target: path to test dataset + # params: + # key: value + # lightning: (optional, has sane defaults and can be specified on cmdline) + # trainer: + # additional arguments to trainer + # logger: + # logger to instantiate + # modelcheckpoint: + # modelcheckpoint to instantiate + # callbacks: + # callback1: + # target: importpath + # params: + # key: value + + now = datetime.datetime.now().strftime("%Y-%m-%dT%H-%M-%S") + + # add cwd for convenience and to make classes in this file available when + # running as `python main.py` + # (in particular `main.DataModuleFromConfig`) + sys.path.append(os.getcwd()) + + parser = get_parser() + parser = Trainer.add_argparse_args(parser) + + opt, unknown = parser.parse_known_args() + if opt.name and opt.resume: + raise ValueError( + "-n/--name and -r/--resume cannot be specified both." + "If you want to resume training in a new log folder, " + "use -n/--name in combination with --resume_from_checkpoint" + ) + if opt.resume: + # if not os.path.exists(opt.resume): + # raise ValueError("Cannot find {}".format(opt.resume)) + if os.path.isfile(opt.resume): + paths = opt.resume.split("/") + # idx = len(paths)-paths[::-1].index("logs")+1 + # logdir = "/".join(paths[:idx]) + logdir = "/".join(paths[:-2]) + ckpt = opt.resume + else: + # assert os.path.isdir(opt.resume), opt.resume + logdir = opt.resume.rstrip("/") + ckpt = os.path.join(logdir, "checkpoints", "last.ckpt") + + if os.path.exists(ckpt): + opt.resume_from_checkpoint = ckpt + base_configs = sorted(glob.glob(os.path.join(logdir, "configs/*.yaml"))) + opt.base = base_configs + opt.base + _tmp = logdir.split("/") + nowname = _tmp[-1] + else: + if opt.name: + name = opt.name + elif opt.base: + cfg_fname = os.path.split(opt.base[0])[-1] + cfg_name = os.path.splitext(cfg_fname)[0] + name = cfg_name + else: + name = "" + nowname = name + opt.postfix + logdir = os.path.join(opt.logdir, nowname) + else: + if opt.name: + # name = "_" + opt.name + name = opt.name + elif opt.base: + cfg_fname = os.path.split(opt.base[0])[-1] + cfg_name = os.path.splitext(cfg_fname)[0] + # name = "_" + cfg_name + name = cfg_name + else: + name = "" + # nowname = now + name + opt.postfix + nowname = name + opt.postfix + logdir = os.path.join(opt.logdir, nowname) + + ckptdir = os.path.join(logdir, "checkpoints") + cfgdir = os.path.join(logdir, "configs") + seed_everything(opt.seed) + + try: + # init and save configs + configs = [OmegaConf.load(cfg) for cfg in opt.base] + cli = OmegaConf.from_dotlist(unknown) + config = OmegaConf.merge(*configs, cli) + lightning_config = config.pop("lightning", OmegaConf.create()) + # merge trainer cli with config + trainer_config = lightning_config.get("trainer", OmegaConf.create()) + # default to ddp + trainer_config["accelerator"] = "ddp" + for k in nondefault_trainer_args(opt): + trainer_config[k] = getattr(opt, k) + if not "gpus" in trainer_config: + del trainer_config["accelerator"] + cpu = True + else: + gpuinfo = trainer_config["gpus"] + print(f"Running on GPUs {gpuinfo}") + cpu = False + trainer_opt = argparse.Namespace(**trainer_config) + lightning_config.trainer = trainer_config + + # model + model = instantiate_from_config(config.model) + + # trainer and callbacks + trainer_kwargs = dict() + + # default logger configs + default_logger_cfgs = { + "wandb": { + "target": "pytorch_lightning.loggers.WandbLogger", + "params": { + "name": nowname, + "save_dir": logdir, + "offline": opt.debug, + "id": nowname, + } + }, + "testtube": { + "target": "pytorch_lightning.loggers.TestTubeLogger", + "params": { + "name": "testtube", + "save_dir": logdir, + } + }, + } + default_logger_cfg = default_logger_cfgs["testtube"] + if "logger" in lightning_config: + logger_cfg = lightning_config.logger + else: + logger_cfg = OmegaConf.create() + logger_cfg = OmegaConf.merge(default_logger_cfg, logger_cfg) + trainer_kwargs["logger"] = instantiate_from_config(logger_cfg) + + # modelcheckpoint - use TrainResult/EvalResult(checkpoint_on=metric) to + # specify which metric is used to determine best models + default_modelckpt_cfg = { + "target": "pytorch_lightning.callbacks.ModelCheckpoint", + "params": { + "dirpath": ckptdir, + "filename": "{epoch:06}", + "verbose": True, + "save_last": True, + } + } + if hasattr(model, "monitor"): + print(f"Monitoring {model.monitor} as checkpoint metric.") + default_modelckpt_cfg["params"]["monitor"] = model.monitor + default_modelckpt_cfg["params"]["save_top_k"] = 3 + + if "modelcheckpoint" in lightning_config: + modelckpt_cfg = lightning_config.modelcheckpoint + else: + modelckpt_cfg = OmegaConf.create() + modelckpt_cfg = OmegaConf.merge(default_modelckpt_cfg, modelckpt_cfg) + print(f"Merged modelckpt-cfg: \n{modelckpt_cfg}") + if version.parse(pl.__version__) < version.parse('1.4.0'): + trainer_kwargs["checkpoint_callback"] = instantiate_from_config(modelckpt_cfg) + + # add callback which sets up log directory + default_callbacks_cfg = { + "setup_callback": { + "target": "main.SetupCallback", + "params": { + "resume": opt.resume, + "now": now, + "logdir": logdir, + "ckptdir": ckptdir, + "cfgdir": cfgdir, + "config": config, + "lightning_config": lightning_config, + } + }, + "image_logger": { + "target": "main.ImageLogger", + "params": { + "batch_frequency": 750, + "max_images": 4, + "clamp": True + } + }, + "learning_rate_logger": { + "target": "main.LearningRateMonitor", + "params": { + "logging_interval": "step", + # "log_momentum": True + } + }, + "cuda_callback": { + "target": "main.CUDACallback" + }, + } + if version.parse(pl.__version__) >= version.parse('1.4.0'): + default_callbacks_cfg.update({'checkpoint_callback': modelckpt_cfg}) + + if "callbacks" in lightning_config: + callbacks_cfg = lightning_config.callbacks + else: + callbacks_cfg = OmegaConf.create() + + if 'metrics_over_trainsteps_checkpoint' in callbacks_cfg: + print( + 'Caution: Saving checkpoints every n train steps without deleting. This might require some free space.') + default_metrics_over_trainsteps_ckpt_dict = { + 'metrics_over_trainsteps_checkpoint': + {"target": 'pytorch_lightning.callbacks.ModelCheckpoint', + 'params': { + "dirpath": os.path.join(ckptdir, 'trainstep_checkpoints'), + "filename": "{epoch:06}-{step:09}", + "verbose": True, + 'save_top_k': -1, + 'every_n_train_steps': 10000, + 'save_weights_only': True + } + } + } + default_callbacks_cfg.update(default_metrics_over_trainsteps_ckpt_dict) + + callbacks_cfg = OmegaConf.merge(default_callbacks_cfg, callbacks_cfg) + if 'ignore_keys_callback' in callbacks_cfg and hasattr(trainer_opt, 'resume_from_checkpoint'): + callbacks_cfg.ignore_keys_callback.params['ckpt_path'] = trainer_opt.resume_from_checkpoint + elif 'ignore_keys_callback' in callbacks_cfg: + del callbacks_cfg['ignore_keys_callback'] + + trainer_kwargs["callbacks"] = [instantiate_from_config(callbacks_cfg[k]) for k in callbacks_cfg] + + trainer = Trainer.from_argparse_args(trainer_opt, **trainer_kwargs) + trainer.logdir = logdir ### + + # data + data = instantiate_from_config(config.data) + # NOTE according to https://pytorch-lightning.readthedocs.io/en/latest/datamodules.html + # calling these ourselves should not be necessary but it is. + # lightning still takes care of proper multiprocessing though + # breakpoint() + data.prepare_data() + data.setup() + + # breakpoint() + # CTReportDataset(data_folder='train', csv_file='/home/v-qichen3/blob/ct_rate/CT-RATE/radiology_text_reports/train_reports.csv') + + print("#### Data #####") + for k in data.datasets: + print(f"{k}, {data.datasets[k].__class__.__name__}, {len(data.datasets[k])}") + + # configure learning rate + bs, base_lr = config.data.params.batch_size, config.model.base_learning_rate + # breakpoint() + if not cpu: + ngpu = len(lightning_config.trainer.gpus.strip(",").split(',')) + else: + ngpu = 1 + if 'accumulate_grad_batches' in lightning_config.trainer: + accumulate_grad_batches = lightning_config.trainer.accumulate_grad_batches + else: + accumulate_grad_batches = 1 + print(f"accumulate_grad_batches = {accumulate_grad_batches}") + lightning_config.trainer.accumulate_grad_batches = accumulate_grad_batches + if opt.scale_lr: + model.learning_rate = accumulate_grad_batches * ngpu * bs * base_lr + print( + "Setting learning rate to {:.2e} = {} (accumulate_grad_batches) * {} (num_gpus) * {} (batchsize) * {:.2e} (base_lr)".format( + model.learning_rate, accumulate_grad_batches, ngpu, bs, base_lr)) + else: + model.learning_rate = base_lr + print("++++ NOT USING LR SCALING ++++") + print(f"Setting learning rate to {model.learning_rate:.2e}") + + + # allow checkpointing via USR1 + def melk(*args, **kwargs): + # run all checkpoint hooks + if trainer.global_rank == 0: + print("Summoning checkpoint.") + ckpt_path = os.path.join(ckptdir, "last.ckpt") + trainer.save_checkpoint(ckpt_path) + + + def divein(*args, **kwargs): + if trainer.global_rank == 0: + import pudb; + pudb.set_trace() + + + import signal + + signal.signal(signal.SIGUSR1, melk) + signal.signal(signal.SIGUSR2, divein) + + # run + if opt.train: + try: + trainer.fit(model, data) + except Exception: + melk() + raise + if not opt.no_test and not trainer.interrupted: + trainer.test(model, data) + except Exception: + if opt.debug and trainer.global_rank == 0: + try: + import pudb as debugger + except ImportError: + import pdb as debugger + debugger.post_mortem() + raise + finally: + # move newly created debug project to debug_runs + if opt.debug and not opt.resume and trainer.global_rank == 0: + dst, name = os.path.split(logdir) + dst = os.path.join(dst, "debug_runs", name) + os.makedirs(os.path.split(dst)[0], exist_ok=True) + os.rename(logdir, dst) + if trainer.global_rank == 0: + print(trainer.profiler.summary()) diff --git a/Abnormal-CT-Generation-MultiDisease/mask_generation.py b/Abnormal-CT-Generation-MultiDisease/mask_generation.py new file mode 100644 index 0000000000000000000000000000000000000000..982545ca730f3ef8f7d0373e7b9a44c7fefb887a --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/mask_generation.py @@ -0,0 +1,292 @@ +### Tumor Generateion +import random +import cv2 +import elasticdeform +import numpy as np +from scipy.ndimage import gaussian_filter + + +def generate_prob_function(mask_shape): + sigma = np.random.uniform(3,15) + # uniform noise generate + a = np.random.uniform(0, 1, size=(mask_shape[0],mask_shape[1],mask_shape[2])) + + # Gaussian filter + # this taks some time + a_2 = gaussian_filter(a, sigma=sigma) + + scale = np.random.uniform(0.19, 0.21) + base = np.random.uniform(0.04, 0.06) + a = scale * (a_2 - np.min(a_2)) / (np.max(a_2) - np.min(a_2)) + base + + return a + +# Step 1: Random select (numbers) location for tumor. +def random_select(mask_scan): + # we first find z index and then sample point with z slice + z_start, z_end = np.where(np.any(mask_scan, axis=(0, 1)))[0][[0, -1]] + + # we need to strict number z's position (0.3 - 0.7 in the middle of liver) + z = round(random.uniform(0.3, 0.7) * (z_end - z_start)) + z_start + + liver_mask = mask_scan[..., z] + + # erode the mask (we don't want the edge points) + kernel = np.ones((5,5), dtype=np.uint8) + liver_mask = cv2.erode(liver_mask, kernel, iterations=1) + + coordinates = np.argwhere(liver_mask == 1) + random_index = np.random.randint(0, len(coordinates)) + xyz = coordinates[random_index].tolist() # get x,y + xyz.append(z) + potential_points = xyz + + return potential_points + +# Step 2 : generate the ellipsoid +def get_ellipsoid(x, y, z): + """" + x, y, z is the radius of this ellipsoid in x, y, z direction respectly. + """ + sh = (4*x, 4*y, 4*z) + out = np.zeros(sh, int) + aux = np.zeros(sh) + radii = np.array([x, y, z]) + com = np.array([2*x, 2*y, 2*z]) # center point + + # calculate the ellipsoid + bboxl = np.floor(com-radii).clip(0,None).astype(int) + bboxh = (np.ceil(com+radii)+1).clip(None, sh).astype(int) + roi = out[tuple(map(slice,bboxl,bboxh))] + roiaux = aux[tuple(map(slice,bboxl,bboxh))] + logrid = *map(np.square,np.ogrid[tuple( + map(slice,(bboxl-com)/radii,(bboxh-com-1)/radii,1j*(bboxh-bboxl)))]), + dst = (1-sum(logrid)).clip(0,None) + mask = dst>roiaux + roi[mask] = 1 + np.copyto(roiaux,dst,where=mask) + + return out + +def get_fixed_geo(mask_scan, tumor_type): + + enlarge_x, enlarge_y, enlarge_z = 160, 160, 160 + geo_mask = np.zeros((mask_scan.shape[0] + enlarge_x, mask_scan.shape[1] + enlarge_y, mask_scan.shape[2] + enlarge_z), dtype=np.int8) + # texture_map = np.zeros((mask_scan.shape[0] + enlarge_x, mask_scan.shape[1] + enlarge_y, mask_scan.shape[2] + enlarge_z), dtype=np.float16) + tiny_radius, small_radius, medium_radius, large_radius = 4, 8, 16, 32 + + if tumor_type == 'tiny': + num_tumor = random.randint(3,10) + for _ in range(num_tumor): + # Tiny tumor + x = random.randint(int(0.75*tiny_radius), int(1.25*tiny_radius)) + y = random.randint(int(0.75*tiny_radius), int(1.25*tiny_radius)) + z = random.randint(int(0.75*tiny_radius), int(1.25*tiny_radius)) + sigma = random.uniform(0.5,1) + + geo = get_ellipsoid(x, y, z) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,1)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(1,2)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,2)) + point = random_select(mask_scan) + new_point = [point[0] + enlarge_x//2, point[1] + enlarge_y//2, point[2] + enlarge_z//2] + x_low, x_high = new_point[0] - geo.shape[0]//2, new_point[0] + geo.shape[0]//2 + y_low, y_high = new_point[1] - geo.shape[1]//2, new_point[1] + geo.shape[1]//2 + z_low, z_high = new_point[2] - geo.shape[2]//2, new_point[2] + geo.shape[2]//2 + + # paste small tumor geo into test sample + geo_mask[x_low:x_high, y_low:y_high, z_low:z_high] += geo + + if tumor_type == 'small': + num_tumor = random.randint(3,10) + for _ in range(num_tumor): + # Small tumor + x = random.randint(int(0.75*small_radius), int(1.25*small_radius)) + y = random.randint(int(0.75*small_radius), int(1.25*small_radius)) + z = random.randint(int(0.75*small_radius), int(1.25*small_radius)) + sigma = random.randint(1, 2) + + geo = get_ellipsoid(x, y, z) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,1)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(1,2)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,2)) + # texture = get_texture((4*x, 4*y, 4*z)) + point = random_select(mask_scan) + new_point = [point[0] + enlarge_x//2, point[1] + enlarge_y//2, point[2] + enlarge_z//2] + x_low, x_high = new_point[0] - geo.shape[0]//2, new_point[0] + geo.shape[0]//2 + y_low, y_high = new_point[1] - geo.shape[1]//2, new_point[1] + geo.shape[1]//2 + z_low, z_high = new_point[2] - geo.shape[2]//2, new_point[2] + geo.shape[2]//2 + + # paste small tumor geo into test sample + geo_mask[x_low:x_high, y_low:y_high, z_low:z_high] += geo + # texture_map[x_low:x_high, y_low:y_high, z_low:z_high] = texture + + if tumor_type == 'medium': + num_tumor = random.randint(2, 5) + for _ in range(num_tumor): + # medium tumor + x = random.randint(int(0.75*medium_radius), int(1.25*medium_radius)) + y = random.randint(int(0.75*medium_radius), int(1.25*medium_radius)) + z = random.randint(int(0.75*medium_radius), int(1.25*medium_radius)) + sigma = random.randint(3, 6) + + geo = get_ellipsoid(x, y, z) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,1)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(1,2)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,2)) + # texture = get_texture((4*x, 4*y, 4*z)) + point = random_select(mask_scan) + new_point = [point[0] + enlarge_x//2, point[1] + enlarge_y//2, point[2] + enlarge_z//2] + x_low, x_high = new_point[0] - geo.shape[0]//2, new_point[0] + geo.shape[0]//2 + y_low, y_high = new_point[1] - geo.shape[1]//2, new_point[1] + geo.shape[1]//2 + z_low, z_high = new_point[2] - geo.shape[2]//2, new_point[2] + geo.shape[2]//2 + + # paste medium tumor geo into test sample + geo_mask[x_low:x_high, y_low:y_high, z_low:z_high] += geo + # texture_map[x_low:x_high, y_low:y_high, z_low:z_high] = texture + + if tumor_type == 'large': + num_tumor = random.randint(1,3) + for _ in range(num_tumor): + # Large tumor + x = random.randint(int(0.75*large_radius), int(1.25*large_radius)) + y = random.randint(int(0.75*large_radius), int(1.25*large_radius)) + z = random.randint(int(0.75*large_radius), int(1.25*large_radius)) + sigma = random.randint(5, 10) + + geo = get_ellipsoid(x, y, z) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,1)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(1,2)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,2)) + # texture = get_texture((4*x, 4*y, 4*z)) + point = random_select(mask_scan) + new_point = [point[0] + enlarge_x//2, point[1] + enlarge_y//2, point[2] + enlarge_z//2] + x_low, x_high = new_point[0] - geo.shape[0]//2, new_point[0] + geo.shape[0]//2 + y_low, y_high = new_point[1] - geo.shape[1]//2, new_point[1] + geo.shape[1]//2 + z_low, z_high = new_point[2] - geo.shape[2]//2, new_point[2] + geo.shape[2]//2 + + # paste small tumor geo into test sample + geo_mask[x_low:x_high, y_low:y_high, z_low:z_high] += geo + # texture_map[x_low:x_high, y_low:y_high, z_low:z_high] = texture + + if tumor_type == "mix": + # tiny + num_tumor = random.randint(3,10) + for _ in range(num_tumor): + # Tiny tumor + x = random.randint(int(0.75*tiny_radius), int(1.25*tiny_radius)) + y = random.randint(int(0.75*tiny_radius), int(1.25*tiny_radius)) + z = random.randint(int(0.75*tiny_radius), int(1.25*tiny_radius)) + sigma = random.uniform(0.5,1) + + geo = get_ellipsoid(x, y, z) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,1)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(1,2)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,2)) + point = random_select(mask_scan) + new_point = [point[0] + enlarge_x//2, point[1] + enlarge_y//2, point[2] + enlarge_z//2] + x_low, x_high = new_point[0] - geo.shape[0]//2, new_point[0] + geo.shape[0]//2 + y_low, y_high = new_point[1] - geo.shape[1]//2, new_point[1] + geo.shape[1]//2 + z_low, z_high = new_point[2] - geo.shape[2]//2, new_point[2] + geo.shape[2]//2 + + # paste small tumor geo into test sample + geo_mask[x_low:x_high, y_low:y_high, z_low:z_high] += geo + + # small + num_tumor = random.randint(5,10) + for _ in range(num_tumor): + # Small tumor + x = random.randint(int(0.75*small_radius), int(1.25*small_radius)) + y = random.randint(int(0.75*small_radius), int(1.25*small_radius)) + z = random.randint(int(0.75*small_radius), int(1.25*small_radius)) + sigma = random.randint(1, 2) + + geo = get_ellipsoid(x, y, z) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,1)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(1,2)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,2)) + # texture = get_texture((4*x, 4*y, 4*z)) + point = random_select(mask_scan) + new_point = [point[0] + enlarge_x//2, point[1] + enlarge_y//2, point[2] + enlarge_z//2] + x_low, x_high = new_point[0] - geo.shape[0]//2, new_point[0] + geo.shape[0]//2 + y_low, y_high = new_point[1] - geo.shape[1]//2, new_point[1] + geo.shape[1]//2 + z_low, z_high = new_point[2] - geo.shape[2]//2, new_point[2] + geo.shape[2]//2 + + # paste small tumor geo into test sample + geo_mask[x_low:x_high, y_low:y_high, z_low:z_high] += geo + # texture_map[x_low:x_high, y_low:y_high, z_low:z_high] = texture + + # medium + num_tumor = random.randint(2, 5) + for _ in range(num_tumor): + # medium tumor + x = random.randint(int(0.75*medium_radius), int(1.25*medium_radius)) + y = random.randint(int(0.75*medium_radius), int(1.25*medium_radius)) + z = random.randint(int(0.75*medium_radius), int(1.25*medium_radius)) + sigma = random.randint(3, 6) + + geo = get_ellipsoid(x, y, z) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,1)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(1,2)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,2)) + # texture = get_texture((4*x, 4*y, 4*z)) + point = random_select(mask_scan) + new_point = [point[0] + enlarge_x//2, point[1] + enlarge_y//2, point[2] + enlarge_z//2] + x_low, x_high = new_point[0] - geo.shape[0]//2, new_point[0] + geo.shape[0]//2 + y_low, y_high = new_point[1] - geo.shape[1]//2, new_point[1] + geo.shape[1]//2 + z_low, z_high = new_point[2] - geo.shape[2]//2, new_point[2] + geo.shape[2]//2 + + # paste medium tumor geo into test sample + geo_mask[x_low:x_high, y_low:y_high, z_low:z_high] += geo + # texture_map[x_low:x_high, y_low:y_high, z_low:z_high] = texture + + # large + num_tumor = random.randint(1,3) + for _ in range(num_tumor): + # Large tumor + x = random.randint(int(0.75*large_radius), int(1.25*large_radius)) + y = random.randint(int(0.75*large_radius), int(1.25*large_radius)) + z = random.randint(int(0.75*large_radius), int(1.25*large_radius)) + sigma = random.randint(5, 10) + geo = get_ellipsoid(x, y, z) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,1)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(1,2)) + geo = elasticdeform.deform_random_grid(geo, sigma=sigma, points=3, order=0, axis=(0,2)) + # texture = get_texture((4*x, 4*y, 4*z)) + point = random_select(mask_scan) + new_point = [point[0] + enlarge_x//2, point[1] + enlarge_y//2, point[2] + enlarge_z//2] + x_low, x_high = new_point[0] - geo.shape[0]//2, new_point[0] + geo.shape[0]//2 + y_low, y_high = new_point[1] - geo.shape[1]//2, new_point[1] + geo.shape[1]//2 + z_low, z_high = new_point[2] - geo.shape[2]//2, new_point[2] + geo.shape[2]//2 + + # paste small tumor geo into test sample + geo_mask[x_low:x_high, y_low:y_high, z_low:z_high] += geo + # texture_map[x_low:x_high, y_low:y_high, z_low:z_high] = texture + + geo_mask = geo_mask[enlarge_x//2:-enlarge_x//2, enlarge_y//2:-enlarge_y//2, enlarge_z//2:-enlarge_z//2] + # texture_map = texture_map[enlarge_x//2:-enlarge_x//2, enlarge_y//2:-enlarge_y//2, enlarge_z//2:-enlarge_z//2] + geo_mask = (geo_mask * mask_scan) >=1 + + return geo_mask + + +def SynthesisTumor(mask_scan, tumor_type, texture): + # for speed_generate_tumor, we only send the liver part into the generate program + x_start, x_end = np.where(np.any(mask_scan, axis=(1, 2)))[0][[0, -1]] + y_start, y_end = np.where(np.any(mask_scan, axis=(0, 2)))[0][[0, -1]] + z_start, z_end = np.where(np.any(mask_scan, axis=(0, 1)))[0][[0, -1]] + + # shrink the boundary + x_start, x_end = max(0, x_start+1), min(mask_scan.shape[0], x_end-1) + y_start, y_end = max(0, y_start+1), min(mask_scan.shape[1], y_end-1) + z_start, z_end = max(0, z_start+1), min(mask_scan.shape[2], z_end-1) + + liver_mask = get_fixed_geo(mask_scan, tumor_type) + + mask_scan[x_start:x_end, y_start:y_end, z_start:z_end] = liver_mask + + return mask_scan + +if __name__ == 'main': + geo_mask = get_fixed_geo(mask_scan, tumor_type) + \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/mask_generation_pipeline.py b/Abnormal-CT-Generation-MultiDisease/mask_generation_pipeline.py new file mode 100644 index 0000000000000000000000000000000000000000..79ec5941dea8f6d790313e38918b79e02bc3597b --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/mask_generation_pipeline.py @@ -0,0 +1,127 @@ +import glob +import nibabel as nib +import numpy as np +import os +import torch + +from einops import rearrange +from omegaconf import OmegaConf +from torch.utils.data import DataLoader +from tqdm import tqdm + +from ldm.util import instantiate_from_config +import argparse + +parser = argparse.ArgumentParser() +parser.add_argument( + "-s", + "--save_path", + type=str, + default=None, +) + +args = parser.parse_args() +ddim_steps=args.time_steps +# breakpoint() + +config = OmegaConf.load('./configs/latent-diffusion/mask_generation.yaml') +data = instantiate_from_config(config.data) +data.prepare_data() +data.setup() + + +save_path = args.save_path +if not os.path.exists(save_path): + os.makedirs(save_path) + +val_dataset = data.datasets['validation'] +batch_size = 1 +valloader = DataLoader(val_dataset, batch_size=batch_size, shuffle=False, num_workers=2, pin_memory=True) +val_num = len(val_dataset) +save_gt = True + +for idx, data in tqdm(enumerate(valloader)): + + if idx >= val_num: + break + + name = data['name'][0] + volume_data = data['volume_data'] + + + window_length = 16 + h = 1 + slice_num =volume_data.shape[1] + result = torch.zeros((batch_size, slice_num, 4, 64, 64)).cuda() + + + upper_iters = (slice_num-h) // (window_length-h)+1 if (slice_num-h)%(window_length-h) != 0 else (slice_num-h) // (window_length-h) + print('upper_iters', upper_iters) + # breakpoint() + for i in range(upper_iters): + print('i', i) + input_data={} + if i == upper_iters-1: + input_data['name'] = data['name'] + input_data['volume_data'] = data['volume_data'][:,-window_length:].to(device) + input_data['masked_data'] = data['masked_data'][:,-window_length:].to(device) + input_data['tumor_mask'] = data['tumor_mask'][:,-window_length:].to(device) + else: + input_data['volume_data'] = data['volume_data'][:, i*window_length-i*h:(i+1)*window_length-i*h].to(device) + input_data['masked_data'] = data['masked_data'][:, i*window_length-i*h:(i+1)*window_length-i*h].to(device) + input_data['tumor_mask'] = data['tumor_mask'][:, i*window_length-i*h:(i+1)*window_length-i*h].to(device) + + with torch.no_grad(): + _, c = model.get_input(input_data, model.first_stage_key) + + if i == 0: + samples_i, _ = model.sample_log(cond=c, batch_size=window_length, ddim=True, eta=1., ddim_steps=ddim_steps) + else: + samples_i, _ = model.sample_log(cond=c, batch_size=window_length, ddim=True, eta=1., ddim_steps=ddim_steps, previous=x_minus1) + # breakpoint() + samples_i = rearrange(samples_i, '(b z) c h w -> b z c h w', z=window_length) + + if i == upper_iters-1: + result[:, -window_length+h:] = samples_i[:,h:,...] + else: + if i == 0: + result[:, :window_length] = samples_i + else: + result[:, i*window_length-i*h+h:(i+1)*window_length-i*h] = samples_i[:, h:] + x_minus1 = samples_i[:, -h:,...] + # breakpoint() + result = rearrange(result, 'b z c h w -> (b z) c h w') + x_result = torch.zeros((result.shape[0],3,512,512)) + # breakpoint() + dec_unit = 16 + num_dec_iter = slice_num // dec_unit + 1 if slice_num % dec_unit != 0 else slice_num // dec_unit + for i in range(num_dec_iter): + if i == num_dec_iter - 1: + x_result[-dec_unit:] = model.decode_first_stage(result[-dec_unit:]) + x_result[i*dec_unit:(i+1)*dec_unit] = model.decode_first_stage(result[i*dec_unit:(i+1)*dec_unit]) + x_result[x_result>1.0] = 1.0 + x_result[x_result<-1.0] = -1.0 + x_result = (x_result+1)/2 + x_result = rearrange(x_result, '(b z) c h w -> b z c h w', z=slice_num) + x_result_ = x_result[0].mean(axis=1).detach().cpu().numpy() + # x_result = x_result[0,:,0,...].detach().cpu().numpy() + + x_result = x_result_.transpose(2,1,0) + # x_result = np.rot90(x_result, k=1, axes=(0,1)) + # x_result = np.flip(x_result,axis=(0,1)) + # import imageio as io + # io.imsave('exp.png', (x_result[:,:,400]*255).astype(np.uint8)) + + # breakpoint() + ref_root = '/storage/chenqi/data/CT/Task03_Liver/imagesTr' + ref_nii = os.path.join(ref_root, name+'.nii.gz') + affine = nib.load(ref_nii).affine + + x_result = x_result*425.0 - 175.0 + data_path = os.path.join(save_path, str(f'{name}.nii.gz')) + data_nii = nib.Nifti1Image(x_result.astype(np.int16), affine) + + nib.save(data_nii, data_path) + + # breakpoint() + diff --git a/Abnormal-CT-Generation-MultiDisease/recon_test.txt b/Abnormal-CT-Generation-MultiDisease/recon_test.txt new file mode 100644 index 0000000000000000000000000000000000000000..e5645cc3c82b8252fa9c75c20f6227c10e8ed8ac --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/recon_test.txt @@ -0,0 +1,5 @@ +kl-f8 +Test average MSE: 229.57958816063248 average PSNR: 25.01263807051269 average SSIM: 0.7587649705198748 + +kl-f4 +Test average MSE: 229.5796621685892 average PSNR: 25.01263423093404 average SSIM: 0.7587650323867727 \ No newline at end of file diff --git a/Abnormal-CT-Generation-MultiDisease/requirements.txt b/Abnormal-CT-Generation-MultiDisease/requirements.txt new file mode 100644 index 0000000000000000000000000000000000000000..432c8e6ef057b8d911781b09493f63e5d2f8e0ce --- /dev/null +++ b/Abnormal-CT-Generation-MultiDisease/requirements.txt @@ -0,0 +1,101 @@ +absl-py==2.3.0 +aiohappyeyeballs==2.6.1 +aiohttp==3.12.12 +aiosignal==1.3.2 +antlr4-python3-runtime==4.8 +async-timeout==5.0.1 +attrs==25.3.0 +blosc2==2.5.1 +Brotli==1.0.9 +cachetools==5.5.2 +certifi==2025.4.26 +charset-normalizer==3.3.2 +contourpy==1.3.0 +cycler==0.12.1 +easydict==1.11 +einops==0.7.0 +filelock==3.17.0 +fonttools==4.58.2 +frozenlist==1.7.0 +fsspec==2025.5.1 +ftfy==6.3.1 +future==1.0.0 +gmpy2==2.2.1 +google-auth==2.40.3 +google-auth-oauthlib==1.2.2 +grpcio==1.73.0 +hf-xet==1.1.3 +huggingface-hub==0.33.0 +idna==3.7 +imageio==2.31.6 +importlib_metadata==8.7.0 +importlib_resources==6.5.2 +Jinja2==3.1.6 +joblib==1.5.1 +kiwisolver==1.4.7 +lazy_loader==0.4 +lightning-utilities==0.9.0 +lpips==0.1.4 +Markdown==3.8 +MarkupSafe==3.0.2 +matplotlib==3.9.4 +mkl_fft==1.3.11 +mkl_random==1.2.8 +mkl-service==2.4.1 +mpmath==1.3.0 +msgpack==1.1.0 +multidict==6.4.4 +ndindex==1.10.0 +networkx==3.2.1 +nibabel==5.2.0 +numpy +oauthlib==3.2.2 +omegaconf==2.1.1 +open-clip-torch==2.7.0 +opencv-python==4.8.1.78 +packaging==25.0 +pandas==2.1.2 +Pillow==10.0.1 +pip==22.3.1 +propcache==0.3.2 +protobuf==4.23.4 +py-cpuinfo==9.0.0 +pyasn1==0.6.1 +pyasn1_modules==0.4.2 +pyDeprecate==0.3.1 +pyparsing==3.1.1 +PySocks==1.7.1 +python-dateutil==2.9.0.post0 +pytorch-lightning==1.4.2 +pytz==2025.2 +PyYAML==6.0.2 +regex==2024.11.6 +requests==2.32.3 +requests-oauthlib==2.0.0 +rsa==4.9.1 +scikit-image==0.22.0 +scikit-learn==1.3.2 +scipy==1.11.3 +seaborn==0.13.0 +setuptools==78.1.1 +six==1.17.0 +sympy==1.13.3 +tensorboard==2.15.1 +tensorboard-data-server==0.7.2 +tensorboardX==2.6.2.2 +test_tube==0.7.5 +threadpoolctl==3.6.0 +tifffile==2024.8.30 +tokenizers==0.13.3 +tqdm==4.66.1 +transformers==4.29.2 +triton==2.1.0 +typing_extensions==4.12.2 +tzdata==2025.2 +urllib3==2.3.0 +volumentations-3D==1.0.4 +wcwidth==0.2.13 +Werkzeug==3.1.3 +wheel==0.45.1 +yarl==1.20.1 +zipp==3.23.0