Official code for Assessing Generative Models via Precision and Recall by Mehdi S. M. Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly, presented at NeurIPS 2018. The poster can be downloaded here.
A list of required packages is provided in requirements.txt and may be installed by running:
pip install -r requirements.txt
If the embedding is computed manually, a minimal set of required packages may be used, see requirements_minimal.txt.
Note that a GPU will significantly speed up the computation of the Inception embeddings, consider installing pip install tensorflow-gpu
.
Example: you have a folder of images from your true distribution (e.g., ~/real_images/
) and any number of folders of generated images (e.g., ~/generated_images_1/
and ~/generated_images_2/
). Note that the number of images in each folder needs to be the same.
/tmp/prd_cache/inception.pb
(Alternate link here. Note that this file needs to be unpacked.)python prd_from_image_folders.py --inception_path /tmp/prd_cache/inception.pb --reference_dir ~/real_images/ --eval_dirs ~/generated_images_1/ ~/generated_images_2/ --eval_labels model_1 model_2
For further customization, run ./prd_from_image_folders.py -h
to see the list of available options.
Example: you want to compare the precision and recall of a pair of generative models in some feature embedding to your liking (e.g., Inception activations).
feats_real
, feats_gen_1
and feats_gen_2
as numpy arrays each of shape [number_of_data_points, feature_dimensions]
.import prd prd_data_1 = prd.compute_prd_from_embedding(feats_real, feats_gen_1) prd_data_2 = prd.compute_prd_from_embedding(feats_real, feats_gen_2) prd.plot([prd_data_1, prd_data_2], ['model_1', 'model_2'])
@inproceedings{precision_recall_distributions,
title = {{Assessing Generative Models via Precision and Recall}},
author = {Sajjadi, Mehdi~S.~M. and Bachem, Olivier and Lu{\v c}i{\'c}, Mario and Bousquet, Olivier and Gelly, Sylvain},
booktitle = {{Advances in Neural Information Processing Systems (NeurIPS)}},
year = {2018}}
External copyright for: prd_score.py prd_score_test.py
inception_network.py
Copyright for remaining files: Mehdi S. M. Sajjadi
License for all files: Apache License 2.0
For any questions, comments or help to get it to run, please don't hesitate to mail us: msajjadi@tue.mpg.de