CLARIN Logo
latest

Introduction

  • About CLARIN:EL
  • How to use this manual
  • Contact & Help

Basic concepts

  • Registration/Sign in
  • Users
  • What can I find in the CLARIN:EL central inventory?
  • When is the content of a resource accessible?
  • Central Inventory Page
  • Menu for registered users

Navigating the infrastructure

  • Browsing
  • Searching
  • Viewing & Downloading
  • Processing

Creating resources

  • The lifecycle of a resource
  • Important information about all LRTs
  • How to create a resource by using the editor
  • How to create a resource by uploading an XML file

Data

  • How to prepare data before uploading
  • Recommended File Formats
    • Guidance on selecting file formats for long-term accessibility and interoperability
      • File Formats for Digital Preservation Policy
      • Format Recommendations

Metadata

  • What are metadata and why are they important?
  • FAIR principles
  • Mandatory metadata
  • General guidelines on metadata
  • Specific guidelines on mandatory metadata
  • Examples of metadata
  • XML metadata descriptions
  • Full schema documentation

Managing resources

  • Actions on resources

Appendix

  • Information on Legal Issues
  • Publications
CLARIN
  • »
  • Recommended File Formats
  • Edit on GitLab

Recommended File Formats¶

Guidance on selecting file formats for long-term accessibility and interoperability¶

This section 1 lists the file formats which are recommended for depositing in CLARIN:EL.

File Formats for Digital Preservation Policy¶

To ensure access and usability of your data to the broadest audience into the long term, the CLARIN:EL team has considered the following factors to determine which file formats are recommended in CLARIN:EL infrastructure:

  • Processability

    • Suitability for the type of resource and/or type of processing.

    • In order to be processable by the CLARIN:EL integrated NLP workflows, textual data have to be in one of the formats that the workflows can process (listed below).

  • Preservation

    • Suitability for research by the designated communities.

    • How widespread the format is: broadly used formats, not deprecated, known to the designated communities.

    • Use of open source rather than proprietary format.

    • Whether the format employs lossy or lossless compression.

The policy, which is based upon the above-mentioned factors, meets the mission of CLARIN:EL to collect, preserve and distribute digital language resources and language processing services for the support of researchers, academics, students, language professionals, citizen scientists and the general public. In order to arrive at the appropriate recommendations for individual file formats, or to decide on their suitability for particular kinds of research activities/types, the purpose for which they are intended has to be considered. For example, while PDF/A has been developed for unproblematic long-term archiving and is an excellent format choice for documentation, it is undoubtedly not suitable for textual data intended for language processing. Therefore, based on the types of resources that are in the scope of the CLARIN:EL user communities and the processes offered/supported, the CLARIN:EL team discerns the following set, pertinent to the field of digital language resources, for which specific recommendations are provided:

  • CLARIN:EL processable data: textual data 2 that can be input data for CLARIN:EL workflows,

  • Textual Data: written unstructured/plain text or originally structured text (e.g., HTML) without linguistic or other mark-up added for research purposes (non-processable by the CLARIN:EL workflows),

  • Text Annotation: annotations of textual source language data, with the original text included or as a stand-off document,

  • Language Description: data that describe a language or some aspect(s) of a language via a systematic documentation of linguistic structures (Grammars, Machine learning (ML) models, Ν-gram models),

  • Lexical/Conceptual Resource: a resource organised on the basis of lexical or conceptual entries (lexical items, terms, concepts etc.) with their supplementary information (e.g., morphological, semantic, statistical information, etc.),

  • Image data: digitized images of analogue sources of written language data for research purposes (e.g., scans of handwriting, photos of inscriptions) οr two-dimensional pictures or figures that are distributed with associated textual data for NLP analysis (e.g., medical images, image data, accompanied with radiological reports, textual data),

  • Audio data: audio recordings providing spoken language data for research purposes (e.g., audio files with the pronunciation of words for a lexicon, recorded interviews, radio broadcasts, etc.),

  • Video data: video recordings providing multimodal or sign language data for research purposes.

Format Recommendations¶

Formats that fulfil the criteria of the Digital Preservation Policy, mentioned above, are preferred; however, additional formats are accepted, as a first-entry level, with the proposal for conversion to recommended formats.

Therefore, file formats are categorized into two preservation levels (recommended, acceptable) always in the context of each case. The acceptable list is not exhaustive, especially in the case of text annotation, but rather indicative, and it is proposed for an acceptable format to be converted to a recommended format.

../../_images/FormatsNew.png
1

The recommendations presented here have been created by the CLARIN:EL technical team to which you can address any suggested updates or questions.

2

See here the guidelines on processable corpora.

Next Previous

© Copyright 2020, CLARIN Technical Team. Revision 8214113b.

Built with Sphinx using a theme provided by Read the Docs.
Read the Docs v: latest
Versions
latest
stable
Downloads
pdf
html
epub
On Read the Docs
Project Home
Builds