Janne Hellsten commited on
Commit
1574d42
1 Parent(s): f7e4867

dataset_tool.py docs update, tf32 disable for calc_metrics.py

Browse files

- Add a justification for using uncompressed zip and uncompressed png (#22)
- Clarify class label usage with dataset_tool.py (#18)
- Disable tf32 in calc_metrics as this has not been tested.

Files changed (2) hide show
  1. calc_metrics.py +2 -0
  2. dataset_tool.py +21 -1
calc_metrics.py CHANGED
@@ -47,6 +47,8 @@ def subprocess_fn(rank, args, temp_dir):
47
  # Print network summary.
48
  device = torch.device('cuda', rank)
49
  torch.backends.cudnn.benchmark = True
 
 
50
  G = copy.deepcopy(args.G).eval().requires_grad_(False).to(device)
51
  if rank == 0 and args.verbose:
52
  z = torch.empty([1, G.z_dim], device=device)
47
  # Print network summary.
48
  device = torch.device('cuda', rank)
49
  torch.backends.cudnn.benchmark = True
50
+ torch.backends.cuda.matmul.allow_tf32 = False
51
+ torch.backends.cudnn.allow_tf32 = False
52
  G = copy.deepcopy(args.G).eval().requires_grad_(False).to(device)
53
  if rank == 0 and args.verbose:
54
  z = torch.empty([1, G.z_dim], device=device)
dataset_tool.py CHANGED
@@ -331,14 +331,34 @@ def convert_dataset(
331
  --source path/ Recursively load all images from path/
332
  --source dataset.zip Recursively load all images from dataset.zip
333
 
334
- The output dataset format can be either an image folder or a zip archive.
335
  Specifying the output format and path:
336
 
337
  \b
338
  --dest /path/to/dir Save output files under /path/to/dir
339
  --dest /path/to/dataset.zip Save output files into /path/to/dataset.zip
340
 
 
 
 
 
341
  Images within the dataset archive will be stored as uncompressed PNG.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
342
 
343
  Image scale/crop and resolution requirements:
344
 
331
  --source path/ Recursively load all images from path/
332
  --source dataset.zip Recursively load all images from dataset.zip
333
 
 
334
  Specifying the output format and path:
335
 
336
  \b
337
  --dest /path/to/dir Save output files under /path/to/dir
338
  --dest /path/to/dataset.zip Save output files into /path/to/dataset.zip
339
 
340
+ The output dataset format can be either an image folder or an uncompressed zip archive.
341
+ Zip archives makes it easier to move datasets around file servers and clusters, and may
342
+ offer better training performance on network file systems.
343
+
344
  Images within the dataset archive will be stored as uncompressed PNG.
345
+ Uncompresed PNGs can be efficiently decoded in the training loop.
346
+
347
+ Class labels are stored in a file called 'dataset.json' that is stored at the
348
+ dataset root folder. This file has the following structure:
349
+
350
+ \b
351
+ {
352
+ "labels": [
353
+ ["00000/img00000000.png",6],
354
+ ["00000/img00000001.png",9],
355
+ ... repeated for every image in the datase
356
+ ["00049/img00049999.png",1]
357
+ ]
358
+ }
359
+
360
+ If the 'dataset.json' file cannot be found, the dataset is interpreted as
361
+ not containing class labels.
362
 
363
  Image scale/crop and resolution requirements:
364