GERTuraX is a series of pretrained encoder-only language models for German.
The models are ELECTRA-based and pretrained with the TEAMS approach on the CulturaX corpus.
In total, three different models were trained and released with pretraining corpus sizes ranging from 147GB to 1.1TB.
The following models are available:
GERTuraX is the outcome of the last 12 months of working with TPUs from the awesome TRC program and the TensorFlow Model Garden library.
Many thanks for providing TPUs!
Made from Bavarian Oberland with ❤️ and 🥨.