Deriving Semantic Validation rules from Industrial Standards: an OPC UA Study

This repository contains all the data required for evaluation of individual sections mentioned in the paper.

5.1 - Constraint extraction from Tables

• This folder contains :
  • The data for analysis of distribution of Table types in all the 28 OPC UA Companion specifications, chosen for the purpose of this task.
  • Code for extracting tables and constraints present in them.
  • An example PDF document of a Companion specification as input data to this code.
  • Output of extracted constraints from this input into excel sheet.

7.3 - Evaluation of Constraint extraction from Tables

• This folder contains all the data required for the evaluation of table and constraint extraction task, like :
  • The Precision Analysis, where we get True positive and False positive values from Table extraction algorithm, by documenting these values everytime when we run a new companion specification on the Table and constraint extraction code provided in 5.1 .
  • These values are provided under column name "Overall Precision from all the tables types in each Companion specification" highlighted in Orange, values of overall precision with respect to every companion specification.(Sometimes there might be a few minor changes in values as the code is under constant improvements)
  • Dataset for Error and inconsistency analysis where the presence of a particular inconsistency marked as Error numbers is denoted with 1 and its absence is denoted with 0.
  • Scripts for the data and visualizations are provided for 7.3 Precision Analysis, 7.3.1 Recall Analysis and 7.3.2 Error Analysis.

7.4 - Evaluation of Constraint extraction from Text

• This folder contains the data for the evaluation constraint extraction from Text task
  • Machinery data used as tranining and testing data for training the Machine Learning models.
  • PackML gold standard data which has annoted information of its sentences as constraints or not constraints, used to test the models trained using the text from an entirely different companion specification that is Machinery.

7.5 - Evaluation of Feasibility with the help of domain experts

• This folder contains all the data for feasibility evaluation task
  • Two batches of results from mturk evaluation campaign
  • Scripts for the result analysis
  • file with input sentences file of domain expert evaluation
  • Merged results of evaluation

SPARQL Rule templates and rule examples.