Ranking Problem Library: RPLIB
RPLIB contains data, solutions, algorithms, and analysis for a wide range of ranking problems. Example data includes ranking problems for sports (e.g., NCAA, NFL), economics, and fairness (e.g., US News and World Report, AirBnB).
Mission of RPLIB
RPLIB extends LOLIB and serves the ranking community, providing data, solutions, algorithms, and comparative benchmarks. We define a ranking problem card similar in motivation to a ML Model Card. Each ranking problem card stores the set of known solutions, optimal values, rankability analysis, algorithm visualizations, and outputs. RPLIB provides an extensible environment for new metric and techniques and provides support for generating structured artificial datasets.
The RPLIB Browser is a hosted web application that provides a graphical interface designed to search, filter, export, and generally explore RPLIB. You may access the browser by visiting https://birg.dev/rplib.
An RPLIB Card is available for each problem instance with associated analysis through the RPLIB Browser. In addition, each RPLIB Card can be examined and extended using an example Jupyter notebook: Click here to start the Jupyter notebook on Google's Colaboratory.
Support the ranking and rankability research community in dataset sharability, data processing, analysis results, and ranking software tools. Collaborate in gathering more data to help in rankability metric development.
RPLIB provides several example notebooks, including an example Jupyter notebook that views a RPLIB problem instance and analysis known as an RPLIB Card. Using this notebook you can access formulae for testing the fairness and ranking bias of a given dataset. Employ these results in determining the best ranking algorithms and extend with custom analysis.
Support researchers through reproducible and transparent access to data and algorithms. We welcome any and all types of contribution and collaboration, including:
Join our open source RPLIB team on GitHub: https://github.com/IGARDS/RPLIB.
Generate new artificial datasets using our sample Jupyter notebook: Click here to open in Colab.