Repository: flox/flox

Issue #1563: feat: simple descriptor parser in rust
Created: 2024-06-11T13:42:20Z
Background
Currently we use temporary_parse_descriptor[1] to parse install descriptors, which appears to parse versions different from how we want them to be parsed for the use with the catalog.
In particular pkgdb parse descriptor tries to determine the use of semver (ranges) or plain versions:
pkgdb parse descriptor 'a.b.c@1.2.3' | jq '.version,semver'
"1.2.3"
null

pkgdb parse descriptor 'a.b.c@=1.2.3' | jq '.version,semver'
"1.2.3"
null

pkgdb parse descriptor 'a.b.c@==1.2.3' | jq '.version,semver'
"=1.2.3"
null

pkgdb parse descriptor 'a.b.c@^1.2.3' | jq '.version,semver'
null
"^1.2.3"

Notably it uses = to denote "exact" matching despite despite = having existing semantics in most semver parsers -- =1.2 matches 1.2.0 .. 1.3.0-0).
In addition, an exact match is requested either by providing something that can not be parsed as semver (e.g.v-im-too-good-for-semver) or by prefixing with = (e.g. =1.2.3).
To normalize this pkgdb strips off the = part.
For catalog based resolution we would prefer "simply" splitting off the version segment and handling any parsing/interpretation within the service. Changing the user input stands in contrast thereof.
[1] 
  
    
      flox/cli/flox-rust-sdk/src/models/manifest.rs
    
    
        Lines 776 to 838
      in
      801c6f3
    
  
  
    

        
          
           /// Parse a shorthand descriptor into structured data 
        

        
          
           /// 
        

        
          
           /// FIXME: this is currently a hack using a tool in `pkgdb` only meant for debugging. 
        

        
          
           pub fn temporary_parse_descriptor(descriptor: &str) -> Result<PackageToInstall, ManifestError> { 
        

        
          
               let output = Command::new(&*PKGDB_BIN) 
        

        
          
                   .arg("parse") 
        

        
          
                   .arg("descriptor") 
        

        
          
                   .arg("--to") 
        

        
          
                   .arg("manifest") 
        

        
          
                   .arg(descriptor) 
        

        
          
                   .output() 
        

        
          
                   .map_err(ManifestError::PkgDbCall)?; 
        

        
          
               let parsed: Result<ParsedDescriptor, _> = serde_json::from_slice(&output.stdout); 
        

        
          
               if let Ok(parsed) = parsed { 
        

        
          
                   let (id, path) = if let Some(mut path) = parsed.pkg_path { 
        

        
          
                       // Quote any path components that need quoting 
        

        
          
                       path.iter_mut().for_each(|attr| { 
        

        
          
                           if attr.contains('.') || attr.contains('"') { 
        

        
          
                               *attr = format!("\"{}\"", attr); 
        

        
          
                           } 
        

        
          
                       }); 
        

        
          
                       let id_part = path.last().cloned().map(Ok).unwrap_or_else(|| { 
        

        
          
                           Err(ManifestError::MalformedStringDescriptor { 
        

        
          
                               msg: "descriptor had an empty path".to_string(), 
        

        
          
                               desc: descriptor.to_string(), 
        

        
          
                           }) 
        

        
          
                       })?; 
        

        
          
                       (id_part, path.join(".")) 
        

        
          
                   } else if let Some(name) = parsed.name { 
        

        
          
                       (name.clone(), name) 
        

        
          
                   } else { 
        

        
          
                       // This should have been caught by `pkgdb parse descriptor`, 
        

        
          
                       // it's more likely that if we hit this we got unexpected output 
        

        
          
                       // that we didn't know how to parse. It's possible, but unlikely, 
        

        
          
                       // and something ignored for the sake of time when we can write 
        

        
          
                       // a parser in `flox` instead of leaning on `pkgdb` debugging tools. 
        

        
          
                       return Err(ManifestError::MalformedStringDescriptor { 
        

        
          
                           msg: "descriptor had no name or path".to_string(), 
        

        
          
                           desc: descriptor.to_string(), 
        

        
          
                       }); 
        

        
          
                   }; 
        

        
          
                   let version = if let Some(version) = parsed.version { 
        

        
          
                       let mut v = "=".to_string(); 
        

        
          
                       v.push_str(&version); 
        

        
          
                       Some(v) 
        

        
          
                   } else { 
        

        
          
                       parsed.semver 
        

        
          
                   }; 
        

        
          
                   let input = parsed.input.map(|input| input.id); 
        

        
          
                   Ok(PackageToInstall { 
        

        
          
                       id, 
        

        
          
                       pkg_path: path, 
        

        
          
                       version, 
        

        
          
                       input, 
        

        
          
                   }) 
        

        
          
               } else { 
        

        
          
                   Err(ManifestError::MalformedStringDescriptor { 
        

        
          
                       msg: String::from_utf8_lossy(&output.stdout).to_string(), 
        

        
          
                       desc: descriptor.to_string(), 
        

        
          
                   }) 
        

        
          
               } 
        

        
          
           } 
        

        
          
            
        
    
  


Acceptance criteria


 implement the simple descriptor parsing as a function or struct in rust


 add tests for the most common cases of descriptors users may provide

 detect missing quotes when > ranges are being used and misinterpreted by the shell as output redirections
 detect empty attr paths
 detect quoted install_ids



 replace temporary_parse_descriptor with new implementation in FromStr impl for PackageToInstall


 document changed behavior for flox install

 catalog behavior in copy of manpage
 explicit requirement to add = when installing with pkgdb



 document version behavior for catalog manifests

Duplicate issue #1484: Don't add `=` for `flox install package@version`
Created: 2024-05-23T18:27:00Z
The manifest man page describes the behavior of version as:
              Requires that the package match either an exact version or a semver range.
              When the first character of the version string is `=' the version must be
              an exact match, otherwise the version string is matched as a semver range.
              Versions that don’t conform to semver must be specified with `='.

This is inaccurate; the = is not required even for versions that don't conform to semver:
azmq.pkg-path = "azmq"
azmq.version = "unstable-2023-03-23"

Meanwhile, the catalog implements the opposite behavior; if a version starts with =, it's expected to conform to semver. The above example resolves fine, but adding an = results in:
Invalid PackageDescriptor (install_id='azmq' attr_path='azmq'
version='=unstable-2023-03-23' allow_pre_releases=None derivation=None): bad version spec:
Invalid simple block '=unstable-2023-03-23'

In my opinion, the catalog behavior makes sense, so we should:

Don't document =, because it's inconsistent with both pkgdb and the catalog
flox install azmq@unstable-2023-03-23 shouldn't add = which it currently does

Closing comment (2024-06-11T15:22:41Z): Closing as dup of #1563