dimanche 13 janvier 2019

Data exchange formats in C++

Introduction

Data exchange formats are a critical building block when multiple platforms and languages interact with each other. Various standards are used in the wild.

Since few years, developers adopted XML and JSON recently as a human readable data format exchange and Google Buffer protocol as a binary data exchange.

However, C++ STL does not offer any support for XML and JSON. One may write a parser (which takes a lot of time), but great libraries are already available.

Data exchange formats

In this post, We are going to take a look at both XML and JSON and how they can handled in C++ (one may take a look at previous post on Google Buffer Protocol).

XML

XML stands for eXtensible Markup Language, standardized since February 1998 by the W3C for storing and transporting data.

Brief introduction to XML syntax

XML relies on tags (as the case for most description languages). But unlike HTML, XML does not use any predefined tags. We are free to create our own tags.

The reader should also remember that XML is used to store and transport data (however HTML focuses on how data should be displayed).

The global syntax of XML is based on hierarchical structure (like HTML) or child and parent-relationship :

An example of XML document would be as shown below :

<?xml version="1.0" encoding="UTF-8"?> <!-- XML prolog -->
<!-- Define root element geeks -->
<geeks>
      <!-- First geek -->
      <geek>
             <name>dennis ritchie</name>
             <year_of_birth>1941</year_of_birth>
             <country_of_birth>USA</country_of_birth>
             <works>
                    <work>C Language</work>
                    <work>UNIX</work>
             </works>
      </geek>

      <!-- Second geek -->
      <geek>
             <name>linus torvalds</name>
             <year_of_birth>1962</year_of_birth>
             <country_of_birth>Finlande</country_of_birth>
             <works>
                    <work>Linux kernel</work>
                    <work>Git</work>
             </works>
      </geek> 

      <!-- Third geek -->
      <geek>
             <name>Eugene Kaspersky</name>
             <year_of_birth>1965</year_of_birth>
             <country_of_birth>Russia</country_of_birth>
             <works>
                    <work>Kaspersky Antivirus</work>
             </works>
      </geek>     
</geeks>
<!-- END OF XML -->

XML in C++

XML can be managed in C/C++ using TinyXML2, an opensource library allowing to parse and create XML documents easily.

One must install TinyXML2 on his/her system before usage :

$ sudo apt-get install libtinyxml2-dev
Parsing XML

Let's parse the content of the XML example provided above using the following code :

#include <iostream>
#include <tinyxml2.h>

using namespace std;
int main(){

    // create main level XML document container
    tinyxml2::XMLDocument xmlDoc;
        
    // load xml file
    if(xmlDoc.LoadFile("geeks.xml") != tinyxml2::XML_SUCCESS)
    {
        cerr << "loading XML failed" << "\n";
        return false;
    }

    // Get reference to root element "geeks"
    tinyxml2::XMLNode* pRoot = xmlDoc.RootElement();
    // Check if pRoot is non empty
    if (pRoot == NULL) return tinyxml2::XML_ERROR_FILE_READ_ERROR;    
    // Display root node    
    cout << "Root Element : " << pRoot->Value() << endl;
    


    // Traverse root element to get it's children    (geek tags in our example) 
    for(tinyxml2::XMLElement* e = pRoot->FirstChildElement(); e != NULL; e = e->NextSiblingElement())
    {      
        cout << "TAG : " << e->Value() << endl;

        // Traverse each geek tag and read it's content
        for(tinyxml2::XMLElement* subEl = e->FirstChildElement(); subEl != NULL; subEl = subEl->NextSiblingElement()){
       
                if(subEl->Value() == string("name"))
                    cout << "Name : " << subEl->GetText() << endl;
                else if(subEl->Value() == string("year_of_birth"))
                    cout << "Year of birth : " << subEl->GetText() << endl;
                else if(subEl->Value() == string("country_of_birth"))
                    cout << "Country of birth : " << subEl->GetText() << endl;
                else if(subEl->Value() == string("works")){
                    cout << subEl->Value() << " : ";                    
                    for(tinyxml2::XMLElement* works = subEl->FirstChildElement(); works != NULL; works = works->NextSiblingElement()){
                            cout << works->GetText() << " \t";                    
                    }
                    cout << endl;
                }
        }
        cout << "------------------------------------" << endl;    
    }

    return 0;
}

Executing the code above yields the following output :

Save to XML

TinyXML2 can also create valid XML documents, a simple demo would be creating an XML file storing pet's names and their respective ages.

#include <iostream>
#include <tinyxml2.h>

using namespace std;
int main(){
    // Create Main level XML container
    tinyxml2::XMLDocument xmlDoc;
    // Add XML prolog 
    xmlDoc.InsertFirstChild(xmlDoc.NewDeclaration());
    

    // Create XML root node called animals 
    tinyxml2::XMLNode* pRoot = xmlDoc.NewElement("animals");
    // Add pRoot to xmlDoc after prolog    
    xmlDoc.InsertEndChild(pRoot);

    // ************* Add first animal to root node *******
    // create an animal tag
    tinyxml2::XMLNode* animalTag_cat = xmlDoc.NewElement("animal");
    pRoot->InsertFirstChild(animalTag_cat);
    
    // add cat's name and age to animal tag
    tinyxml2::XMLElement* animalName_cat = xmlDoc.NewElement("name");
    // Set animal kind and name
    animalName_cat->SetAttribute("type", "cat");
    animalName_cat->SetText("Oscar"); 
    // Insert cat's name as first child of animal    
    animalTag_cat->InsertFirstChild(animalName_cat);
    // Set cat's age    
    tinyxml2::XMLElement* animalAge_cat = xmlDoc.NewElement("age");
    animalAge_cat->SetText(3); 
    // Insert cat's age as last child of animal    
    animalTag_cat->InsertEndChild(animalAge_cat);   

    // ************* Add second animal to root node *******
    tinyxml2::XMLNode* animalTag_Dog = xmlDoc.NewElement("animal");
    pRoot->InsertEndChild(animalTag_Dog);
    tinyxml2::XMLElement* animalName_dog = xmlDoc.NewElement("name");
    animalName_dog->SetAttribute("type", "dog");
    animalName_dog->SetText("Ace"); 
    animalTag_Dog->InsertFirstChild(animalName_dog);
    tinyxml2::XMLElement* animalAge_dog = xmlDoc.NewElement("age");
    animalAge_dog->SetText(5); 
    animalTag_Dog->InsertEndChild(animalAge_dog);


    // Write xmlDoc into a file
    xmlDoc.SaveFile("animals.xml");
  
    return 0;
}

which produces the output shown below :

JSON

JSON stands for JavaScript Object Notation, a lightweight data exchange format created to replace XML. It became quickly the 1st choice for data transition and transport.

Brief introduction to JSON syntax

JSON has an easier syntax compared to XML, in fact; it looks like Python dictionaries. An example is shown below :

{
    "game_name" : "super mario bross",
    "release_year" : 1985,
    "company" : "Nintendo",
    "developers" : [
                        {"developer" : "Shigeru Miyamoto"}, 
                        {"developer" : "Takashi Tezuka"}
                   ]
}

JSON in C++

Various libraries can be used to parse JSON, though; We are going to focus on the most widely used JsonCpp. It's package can be installed as follow:

$ sudo apt-get install libjsoncpp-dev
Parsing JSON

The following example parses the JSON file providing above (which carries information about super mario bross) :

#include <iostream>
#include <fstream>
#include <jsoncpp/json/json.h>

using namespace std;

int main() {
    // Create an input file stream and load games.json's content
    ifstream ifs("game.json");

    // Create a JSON Reader
    Json::Reader reader;
    Json::Value jsonContentHolder;
    // Parse JSON and load result into obj
    reader.parse(ifs, jsonContentHolder);

    // Display Parser content
    cout << "Game : " << jsonContentHolder["game_name"].asString() << endl;
    cout << "Release Year : " << jsonContentHolder["release_year"].asUInt() << endl;
    cout << "Company : " << jsonContentHolder["company"].asString() << endl;
    
    // create Json::Value object to parse obj["developers"]
    const Json::Value& developers = jsonContentHolder["developers"];
    for (int i = 0; i < developers.size(); i++)
        cout << "----------------- developer : " << developers[i]["developer"].asString() << endl;

    return 0;
}
Executing the above code should yield to the output shown below:
Save to JSON
Writing valid JSON files is also easy using jsoncpp, we provide the example of creating a JSON file storing some countries world's population, population growth rate and country area for year 2018 (information were taken from http://worldpopulationreview.com/countries/).
#include <iostream>
#include <fstream>
#include <jsoncpp/json/json.h>

using namespace std;

int main() {
    
    // Create Json::StyledWriter object to write a JSON file
    Json::StyledWriter styled;

    // JSON content Holder
    Json::Value countriesPopulation;  

    // Add china to countriesPopulation
    countriesPopulation["countries"]["China"]["population"] = 1415045928;
    countriesPopulation["countries"]["China"]["population_percentage_growth_rate"] = 0.39;
    countriesPopulation["countries"]["China"]["country_area_km_square"] = 9706961;

    // Add india to countriesPopulation
    countriesPopulation["countries"]["India"]["population"] = 1354051854;
    countriesPopulation["countries"]["India"]["population_percentage_growth_rate"] = 1.11;
    countriesPopulation["countries"]["India"]["country_area_km_square"] = 3287590;

    // Add france to countriesPopulation
    countriesPopulation["countries"]["France"]["population"] = 65233271;
    countriesPopulation["countries"]["France"]["population_percentage_growth_rate"] = 0.39;
    countriesPopulation["countries"]["France"]["country_area_km_square"] = 551695;

    // Add algeria to countriesPopulation
    countriesPopulation["countries"]["Algeria"]["population"] = 42008054;
    countriesPopulation["countries"]["Algeria"]["population_percentage_growth_rate"] = 1.67;
    countriesPopulation["countries"]["Algeria"]["country_area_km_square"] = 2381741;

    // Create a formatted JSON string
    string sStyled = styled.write(countriesPopulation);

    // Display JSON String
    std::cout << sStyled << std::endl;

    // Write JSON string into a file
    ofstream out("world_population.json", ofstream::out); 
    out << sStyled;
    out.close();    
 
    return 0;
}

Executing the code gives the following :