WikiFactDiff is a dataset designed as a resource to perform atomic factual knowledge updates on language models, with the goal of aligning them with current knowledge. It describes the evolution of factual knowledge between two dates, named T_old and T_new, in the form of semantic triples. To enable the possibility of evaluating knowledge algorithms (such as ROME, MEND, MEMIT, etc.), these triples are verbalised and neighbour facts are determined to check for eventual overflow.
The GitHub project under MIT licence can be used for two purposes:
- Build an instance of WikiFactDiff given two dates T_old and T_new
- Evaluate knowledge update algorithms on a WikiFactDiff instance