Package: m3inference
M3-InferenceThis is a PyTorch implementation of the M3 (Multimodal, Multilingual, and Multi-attribute) system described in the WebConf (WWW) 2019 paperDemographic Inference and Representative Population Estimates from Multilingual Social Media Data.Quick LinksAboutInstallFAQsCitationContactLicenseAboutM3 is a deep learning system for demographic inference that was trained on a massive Twitter dataset. It features three major attributes:MultimodalM3 takes both vision and text inputs. Particularly, the input may contain a profile image, a name (e.g., in the form of a natural language first and last name), a user name (e.g., the Twitter screen_name), and a short self-descriptive text (e.g., a Twitter biography).MultilingualM3 operates in 32 major languages spoken in Europe, but note that these are not all "European" languages (e.g., Arabic is supported). They are['en', 'cs', 'fr', 'nl', 'ar', 'ro', 'bs', 'da', 'it', 'pt', 'no', 'es', 'hr', 'tr', 'de', 'fi', 'el', 'ru', 'bg', 'hu', 'sk', 'et', 'pl', 'lv', 'sl', 'lt', 'ga', 'eu', 'mt', 'cy', 'rm', 'is', 'un']inISO 639-1 two-letter codes(unstands for languages that are not in the list). Alist with the full names of the languages is on the wiki.Multi-attributeThanks to multi-task learning, the model can predict three demographic attributes (gender, age, and human-vs-organization status) at the same time.InstallTL;DRpip install m3inferenceIf there is an error with the installation oftorch, you may install it withconda(seehere). Alternatively, you could create a conda environment - see instructions below.Please ensure you have Python 3.6.6 or higher installed.Manually InstallWith pipYou must havePython>=3.6.6andpipready to use. Then you can:Install dependency packages:pip install -r requirements.txtInstall the packagepython setup.py installAs a conda environmentSimply runconda-env create -f env_conda.yml, you should then have a "m3env" environment available which you can enter withconda activate m3env. Run everything else from within there.Install the packagepython setup.py installHow to useWith M3M3 takes an input of ajsonlfile that containsa list of json(dict) objects(or a python object containing the data itself) and outputs the predictions for the three attributes.Demo withtestdir:Clone this package (git clone https://github.com/zijwang/m3inference.git) and followManually Installto install the package.Preprocessthe image to get them resized to the correct shape. To do this, at the same (root) dir, runpython scripts/preprocess.py --source_dir test/pic/ --output_dir test/pic_resized/ --jsonl_path test/data.jsonl --jsonl_outpath test/data_resized.jsonl --verboseYou may also runpython scripts/preprocess.py --helpto see detailed usages. Further, seeFAQsfor more information on images.In Python, run:from m3inference import M3Inference
import pprint
m3 = M3Inference() # see docstring for details
pred = m3.infer('./test/data_resized.jsonl') # also see docstring for details
pprint.pprint(pred)You should see results like the following:OrderedDict([('720389270335135745',
              {'age': {'19-29': 0.1546,
                       '30-39': 0.114,
                       '<=18': 0.0481,
                       '>=40': 0.6833},
               'gender': {'female': 0.0066, 'male': 0.9934},
               'org': {'is-org': 0.7508, 'non-org': 0.2492}}),
             ('21447363',
              {'age': {'19-29': 0.0157,
                       '30-39': 0.9837,
                       '<=18': 0.0004,
                       '>=40': 0.0002},
               'gender': {'female': 0.9866, 'male': 0.0134},
               'org': {'is-org': 0.0002, 'non-org': 0.9998}}),
    ...
  ...Each entry of the input file (./test/data.jsonl) should have the following keys:id,name,screen_name,description,lang,img_path.The first four keys could be extracted directly from the Twitter JSON entry.Forlang, even if the official Twitter JSON entry contains this field, we recommend to try to use ourcld2wrapper method (from m3inference import get_lang) to get the language from either user's biography/description or the user's tweets. You could also hard-code the language if you know the ground truth from other sources.Images should be downloaded from Twitter as 400x400 pixel images and resized to 224x224 pixels using the preprocess code above.The output file is a dict in which theids are the keys and the predictions are the nested values. The values represents the probability of that category ([0, 1]).For other model settings (e.g., output format, GPU setting, batch_size, etc.), please use the filetest/data.jsonlas a sample input file and see the docstrings foM3Inferenceinitialization and infer method for detailed utilization.With M3 Twitter WrapperExisting JSON Twitter dataIf you have a Twitter JSON object representing a user but donothave images ready, you can use ourM3Twitterclass to:Download and resize the imagesAdd a detected language using CLD2 over the biography textTransform the JSON into the input structure required for M3.from m3inference import M3Twitter
import pprint

m3twitter=M3Twitter(cache_dir="twitter_cache") #Change the cache_dir parameter to control where profile images are downloaded
m3twitter.transform_jsonl(input_file="test/twitter_cache/example_tweets.jsonl",output_file="test/twitter_cache/m3_input.jsonl")

pprint.pprint(m3twitter.infer("test/twitter_cache/m3_input.jsonl")) #Same method as M3Inference.infer(...)If you already have images locally, please include theimage_path_keyparameter and set it to the key in your JSON object containing the path to the image locally. Similarly, if you have detected languages, you can use thelang_keyparameter. An example is given intest/test_transform_jsonl.pyNothing but a screen_name or numeric idYou can also run the Twitter wrapper directly for a Twitter screen_name or numeric id.Please download the "scripts" folders from this repository.To run these examples, you need Twitter API credentials. Please create a Twitter app athttps://developer.twitter.com/en/apps. Once you have an app, copyscripts/auth_example.txttoauth.txtand insert the API key, API secret, access token, and access token secret into this file.Then you can run the following commands:#If you have a screen_name, use
$ python m3twitter.py --screen-name=computermacgyve --auth auth.txt --skip-cache

#If you have a numeric id, use
$python m3twitter.py --id=19854920 --auth auth.txt --skip-cacheThe--skip-cacheoption ensures fresh results are retrieved rather than served from the cache. This is great for debugging but not in a real-world setting; so, remove as needed.FAQsWhat if I just have a Twitter screen name or id?You can use the M3Twitter class to get all the needed profile information (and image) from the Twitter website. Please note this function should only be used for a small number of screen_names or numeric ids. If you have a large list, please use the Twitter API to get the required information (apart from the profile photo, which can be downloaded separately using the.transform_jsonl(...)method described above).import pprint
from m3inference import M3Twitter
m3twitter=M3Twitter()

# initialize twitter api
m3twitter.twitter_init(api_key=...,api_secret=...,access_token=...,access_secret=...)
# alternatively, you may do
m3twitter.twitter_init_from_file('auth.txt')

pprint.pprint(m3twitter.infer_id("2631881902"))The.infer_screen_name(...)method does the same for a Twitter screen name. All results are stored/cached in "~/m3/cache/". This directory can be changed in the M3Twitter constructor and you can skip/update the cache for a single request by settingskip_cache=Trueon the.infer_id(...)or.infer_screen_name(...)method.You can also run these examples directly from the terminal to try things out:python scripts/m3twitter.py --screen-name=barackobama --auth auth.txtHow should I get the images?If you have nothing that a screen name or numeric id, you can use theM3Twitter.infer_screen_name(...)orM3Twitter.infer_id(...)methods. Please note, however, that these methods directly access the Twitter website, not the API and therefore are suitable only to small lists. With a large list of screen_names/ids, please use the Twitter API to get user information.Once you have Twitter JSON, you can use theM3Twitter.transform_jsonl(...)to download images, run language detection, and transform the data to the M3 input format.What if I cannot have image data?In the package, we do provide the standalonetext-based model. You could setuse_full_model=Falsewhen initializingM3Inferenceobject (i.e.,m3=M3Inference(use_full_model=False)). You then do not need to provideimg_pathfield in the input json file.Warning: as M3 model is optimized with the best performance when both image and text inputs are available. You may experience lower performance when using thetext-based model. We recommend using image data whenever possible to get the most accurate predictions.CitationPlease cite our WWW 2019 paper if you use this package in your project.@inproceedings{wang2019demographic,
  title={Demographic inference and representative population estimates from multilingual social media data},
  author={Wang, Zijian and Hale, Scott and Adelani, David Ifeoluwa and Grabowicz, Przemyslaw and Hartman, Timo and Fl{\"o}ck, Fabian and Jurgens, David},
  booktitle={The World Wide Web Conference},
  pages={2056--2067},
  year={2019},
  organization={ACM}
}More QuestionsWe use issues on this GitHub for all questions or suggestions.  For specific inqueries, please contact us atm3@euagendas.org.  Please note that we are unable to release or provide training data for this model due to existing terms of service.LicenseThis source code is licensed under the GNU Affero General Public License, which allows for non-commercial re-use of this software.  For commercial inqueries, please contact us directly. Please see the LICENSE file in the root directory of this source tree for details.