Ranking Problem Library: RPLIB
RPLIB contains data, solutions, algorithms, and analysis for a wide range of ranking problems. Example data includes ranking problems for sports (e.g., NCAA, NFL), economics, and fairness (e.g., US News and World Report, AirBnB).
Mission of RPLIB
RPLIB extends LOLIB and serves the ranking community, providing data, solutions, algorithms, and comparative benchmarks. We define a ranking problem card similar in motivation to a ML Model Card. Each ranking problem card stores the set of known solutions, optimal values, rankability analysis, algorithm visualizations, and outputs. RPLIB provides an extensible environment for new metric and techniques and provides support for generating structured artificial datasets.
RPLIB Browser
The RPLIB Browser is a hosted web application that provides a graphical interface designed to search, filter, export, and generally explore RPLIB. You may access the browser by visiting https://birg.dev/rplib.
RPLIB Card
An RPLIB Card is available for each problem instance with associated analysis through the RPLIB Browser. In addition, each RPLIB Card can be examined and extended using an example Jupyter notebook: Click here to start the Jupyter notebook on Google's Colaboratory.
Contributing New Data
RPLIB is designed to support and engage the community. If you have a dataset you would like added, fill out this form.
Support the ranking and rankability research community in dataset sharability, data processing, analysis results, and ranking software tools. Collaborate in gathering more data to help in rankability metric development.
RPLIB provides several example notebooks, including an example Jupyter notebook that views a RPLIB problem instance and analysis known as an RPLIB Card. Using this notebook you can access formulae for testing the fairness and ranking bias of a given dataset. Employ these results in determining the best ranking algorithms and extend with custom analysis.
Building Community
Support researchers through reproducible and transparent access to data and algorithms. We welcome any and all types of contribution and collaboration, including:
Join our open source RPLIB team on GitHub: https://github.com/IGARDS/RPLIB.
Contribute a dataset: https://docs.google.com/forms/d/e/1FAIpQLSenO1WO_LlzNQ1ak4IPyOjBKkuixZU93umLgeI2kJbFxwzcZQ/viewform
Generate new artificial datasets using our sample Jupyter notebook: Click here to open in Colab.