Posts Tagged synchronization

Informix Application Limitations – Part 3

Although we tried to fix as many of the limitations as possible, we still have a few remaining. We will consider these for future enhancements. Some of these limitations can be fixed with a more advanced web service.

1. User defined data types and binary types are not supported

  • The web service has no way of retrieving the data from user defined data types. Therefore, what we do instead is we determine the column type and based on that, if it is not a built-in data type, we ignore that column. We do this because otherwise, the application wouldn’t even print the information of the rest of the columns. Instead, what the user will see now is all the information stored in the table except for the columns that contain user defined data types.
  • The user can also insert/update rows but only columns with built-in data types will be displayed. The user will also have the ability to delete rows.
  • Like we said though, this is a limitation from the web service and not necessarily of the application itself. So perhaps by developing a more advanced web service you might be able to solve this problem. If you do, then feel free to let us know in the comments section.

2. Unique Constraint

  • There needs to be a column that is unique. This is important because the whole structure of the application is based around the fact that there is a unique key. We use this unique key to organize the details, to determine if there are duplicates or not, and for conflict resolution during insert/update/delete operations.
  • The user must specify which is the unique key. If the user specifies an incorrect unique key, the application won’t work properly.

3. Magic Quotes need to be off in your PHP web service

  • What magic quotes does is protect your server from hacks. It isn’t really that effective though and it can be annoying sometimes. So annoying, that it won’t be in the next version of PHP. But for now, you need to turn it off because when you send the SQL statement to the web service, if it contains quotations (“ ”) it adds backslashes and then the SQL statement won’t work.

4. The row can’t have columns with special characters (i.e. &)

  • We tried to solve this one and we believe it is a problem with the web service as well but we are not completely certain what the problem seems to be. If there is an element that contains special characters though, it won’t update or insert or delete. The application basically does nothing with this kinds of element. Of course, when we say that it won’t do anything with them, we are referring to the Informix Database. It does update, insert and what not in Core Data. When it tries to do the syncing though, that’s when the problem appears.

Miscellaneous

- We had to change the way the Informix Server handles date because PHP returns dates in the Y4MD format whereas the default setup in the Informix Server is MDY2 (or MDY4). Therefore, if the user tries to insert a new row with the Y4MD format, the server would complain. To fix this, we changed the Informix Server date format to Y4MD by setting the DBDATE environment variable.

- There are some columns that are serialized. This means that Informix assigns a value to this columns and they are all unique. Because of this, we don’t let the user to edit or insert data into serial columns. What we do to handle this is that we check the type of the column and if it is serialized then we don’t let the user insert or edit it. In Core Data, when the user inserts a new row (unlike Informix, Core Data doesn’t have the function to add a serial value automatically) we find the highest serial value in the column, and increment the value by one for the new row.

- If the server returns an error (because an operation couldn’t be performed) we catch this error and we place it in an XML file, then the php web service exits and the application grabs this error and presents it to the user.

- In the application we have a details class (.h, .m) that is used to store the details of a specific table. With this class, which is used as reference to create a managedObject, we store the tableName, columns, columnTypes and the number of columns in the table. We also store the uniqueKey (be it user selected or because there is a Unique Constraint within the table itself) and it’s index. This index we use it so we know which column is the uniqueKey; the index just points to the column. We use this information in case there is no connection the Informix database, we can successfully create the GUI and run the application without any issues.

If you have not already downloaded the source code for this demo app, you can download it here.

September 14, 2011

Informix Application Contents – Part 2

Before reading this, please read the first part of this article series to make sure you have the required knowledge to continue. Thank you and we hope this article helps you understand the source code better.

I. Behind the Scenes

Here we will explain what is in the code and what each class contains. We won’t go over all the code, as that would probably require a whole book, but we will introduce each class so you have a simple understanding of their functionality. That way, in case you want to change something specific, you know where to look.

-Under the TBXML Folder

Here you will find two classes (NSDataAdditions and TBXML). This is the open source code we talked about in the first part of this article series. We didn’t change anything within these two classes and you shouldn’t either. We used it as it was provided.

These two classes are what we use to parse through the XML file we receive from the web service. They contain different sets of functions that allow us to easily parse through the XML file and grab the desired information.

-Under the Classes Folder

This is, as you might have guessed, the most extensive folder. This will take a while but we will explain all the classes so you can understand their functionality.

CustomGUIAppDelegate(.h, .m)

Assuming you have some knowledge in Objective-C, you will know that this is where all the main functions occur. This is where the views are loaded when the application launches and where everything else is closed when the application closes.

Inside this code not only do we load the views (which we will explain in detail later) but we also create our managedObjectModels and managedObjectContexts. We have actually created one of them dynamically based on what is grabbed from the database and not with an xcdatamodel, but you will find that it is not that complicated to create them programmatically. On the other hand, if you have created these programmatically before, then this won’t be new to you except for the fact that we actually use two managedObjectModels and two managedObjectContexts (as well as two PersistentStoreCoordinators) which might seem a bit odd. Why do we do this? Pretty simple. In the first one (the one with the xcdatamodel), we store everything regarding the table content. That is, we store the information about the table in the database (i.e. if we are using a customer table, we store the column names, column types, etc). This managedObjectModel has a relationship attribute to another table called “attributes”. This is a to-many relationship and we do this so we can actually store as many attributes for a row as we want. Since it is all under the same database, some of the tables have more attributes (columns) than the other, and if we didn’t have a to-many relationship and instead we had a specific amount of attributes assigned, then we could run short of attributes. But you might be asking yourself, what happens if there is no connection to the database? Good question. That is where the second managedObjectModel comes into play.

In the second managedObjectModel (which we created programmatically), we store the information regarding the table itself. That is, we store the column names, the amount of columns and everything regarding the table. That way, if the application launches and there is no connection (therefore there is no way to get the information about the table regarding the number of columns, column names, column types, etc.) we can use this managedObjectModel determine the number of columns, column names etc. from the data that was already stored in Core Data, so we can once again create the first managedObjectModel.

Take note, that in order for the application to do this, it must have connected to the database at least once in the lifetime of the application, or else it would just be impossible to get this information.

CDViewController (.h, .m)

This is where everything related to Core Data happens. Here, not only do we have the code for the Core Data view itself (that is, the view that shows what we have in Core Data) but we also have the code to add and delete information from Core Data as well.

ColNamesAndTable(.h, .m)

As the name suggests, we use this table to store the column names (and information about the column such as data type and length) and the tableName. That way we can later find which tableName we are currently using and based on that we can determine which are the columns that are applicable to that tableName.

We also have a method that returns the number of columns the table contains. We use this method to return that value to the AppDelegate so we can use it for loops (and other operations) in other classes.

dbViewController (.h, .m)

This class simply takes care of the Informix database view. Basically, this class gathers the information from the Informix database and displays it to the user.

GenericTable(.h, .m)

This class is pretty simple and there is really nothing to it. It only contains a NSMutableArray called attributes that we use to store the attributes of a specific element. We did this in a new class as a way to keep things a little bit more organized but this array could have just as well been added to another class and this class wouldn’t be necessary.

PickTableName (.h, .m)

This class controls the first view the user sees. What we have here is a pickerView, which is the little scrolling wheel that is familiar to the iPhone users. There is a method that gathers the table names from the database (using one of the generic web services we have created) and stores them in an NSMutableArray. This NSMutableArray is later used to display the table names in the pickerView. Using the code provided by Apple already, we can easily know which table is being selected from the pickerView and we store that as theTableName (the variable from the appDelegate).

SyncViewController (.h, .m)

This controls the syncing view but more than that, it controls the syncing itself. It has two methods for the synchronization. It has one for Core Data, where it verifies if the row is already in Core Data, and if it is not, then it inserts it (by calling on a method from CDViewController).

The other method takes care of syncing from Core Data into Informix. It checks if there are any new rows in Core Data. If there are new rows, it inserts them into the Informix database. Then it checks if any of the rows have been updated in Core Data, if they have been, it updates it in the Informix database. And last, but not least, it checks if any of the rows have been deleted, if they are, it deletes them from the Informix database.

TextFieldViewControler (.h, .m)

This class takes care of the insert view. Apart from that, it also has a few methods that work with the SyncViewController class. Here you will find the methods that take care of adding new rows into Informix, as well as the method that updates and deletes rows in the Informix database.

UpdateCoreDataViewController (.h, .m)

This class takes care of the view when the user clicks on an element in the Core Data view. What it displays is editable text fields so the user can edit that data that specific row already contains. It also takes care of the unique key. That is, it detects which column is the unique key and it doesn’t make it editable, because since it is unique, it should be editable by only a database administrator or some privileged users.

UniqueKeyViewController (.h, .m)

This class functions very similarly to the PickTableName class. It displays a pickerView and the information that is displayed in the pickerView is the column names. The column names are gathered from the Informix catalog table via the web service and stored in a NSMutableArray. Then based on the code provided by Apple, we can determine which column the user selected and we set it as the unique key. We also assign this value to the variable uniqueKey from the appDelegate. That way we can use that unique key in other classes for further operations.

ViewDBDetails (.h, .m)

This class controls what the user sees when they click on a row in the database view. Basically it just displays (in a non-editable way) all the data that the row contains. It looks a lot like the UpdateCoreDataViewController except the fields are not editable.

-Under the Other Sources Folder

We didn’t modify these classes at all, and you shouldn’t have to do so either.

-Under the Resources Folder

Attributes(.h, .m)

This is the class we use to store the attributes information. We store the attributeName (which is just the column name) and the attributeValue.

Event (.h, .m)

This is the class that contains the “attributes” for the Informix Database information. It is the one we use as reference for Core Data, in order to store the information from the Informix Database. There are also two internal attributes defined: isNew and edited. We use these attributes to determine whether the row in Core Data was added from the phone (therefore it is new) and if it was updated. We then use these values to know what to do when we sync with the Informix Database.

We also have an attributes relationship, which we connect to another table so we can store the attributes themselves. This is, as previously mentioned, a to-many relationship.

Details (.h, .m)

This class is similar to the Event class, except we use this class as a reference for the table details. This is the class we use to store the columns names, number of columns, column type, and table name.

CustomGUI (.xcdatamodel)

This is the data model we use to store the content within the table.

.

.

PickTableName (.xib) and UniqueKeyViewController (.xib)

These files are views made with Interface Builder. All they contain is a pickerView and the connection to its respective methods so it knows what to print within the pickerView. These methods are taken from their respective classes (PickTableName with PickTableName.h and .m, and UniqueKeyViewController with UniqueViewController.h and .m).

-Misc

libz.dylib

As you can see, this package has been added and it doesn’t belong to any of the previously mentioned folders. We need this package so the TBXML code can actually work. Although the package will already be there once you have downloaded our source code, here is how to add it to your project in case you want to start your own project from scratch:

1. Expand the “Targets”.

2. Right click on the file that appears, it should be the name of your project.

3. Then click on Add > Existing Frameworks…

4. In the new window, there is a + sign on the left bottom corner. Click on it.

5. Scroll down to the very bottom and you will find libz.dylib.

6. Select it and click add.

7. Then close the window and it should have been added.

II. The Views

.

.

1.Pick the table

This is the first view the user has. Here they have a pickerView that will let them choose whichever table they desire to work with.
.
.
.

.

.

.

2.Pick The Unique Column

This view comes right after the user has selected the desired table. Here the pickerView will show the columns that belong to that specific table. Based on that, the user needs to select the column that serves as the unique key. Note: this view only comes up if the table doesn’t have a primary key constraint. If it does, the application automatically selects the Unique key.

.
.

.Below we will see the rest of the views

.

1. The first thing the user will see once he/she has selected the unique key is the database view. Here they will see the rows of the selected table that are in the Informix database. They can click on whichever element (row in the table) they desire and this will take them to a new view that will display the column names and the data in the row (it will display all the columns/attributes and their corresponding values).

.

.

.

2. The second view the user will have available (the second tab at the bottom of the screen) is the insert view. In this view the user can insert a new element (a table row). This new element can either be inserted directly to Informix database or it can be inserted to Core Data. Note: this view is generated once the user has chosen the table and the unique key. Based on that information, the application gathers the column names and generates the insert view.

.

.

3. The third view (the third tab at the bottom of the screen) is for the Core Data view. Here the user can see the data that has already been stored in Core Data. It is pretty similar to the Database view, except this one contains the Core Data view instead. The user can also click on whichever element he/she desires. This time, the view that comes up, lets the user edit every single attribute except for the unique key attribute. All the user has to do is just edit the data. The changes will be stored in Core Data automatically. By going back to the Core Data view the user will notice the changes.

.

4. The fourth and last view (the fourth and last tab) contains the sync buttons. These two buttons let the user manually sync the new information from either Informix to Core Data or from Core Data to Informix. It is worth noting that once the user opens the application for the first time, all the information from Informix is synced into Core Data and every time the user closes the application, all the information from Core Data will be synced into Informix (that is, if there is something new to sync). Then after the application is opened again, the application checks if there are any changes from Core Data that weren’t saved the last time and it syncs them into Informix and then it checks if there is something new from Informix and syncs it into Core Data. Then the process just keeps repeating itself with every execution and closing of the application.

By now you must be eager to get your hands on the source code. However, we have one more article in this three-part series. The source code can be downloaded here.

September 13, 2011

Introduction to the Informix Demo Application – Part 1

What is the application all about?

The goal of this iPhone application is to provide the iPhone app developers an open-source demo app that shows what could be done with Informix database server and the iPhone. The main objective is to let the users access the data from their database and edit the data right from the phone, without having the need to be connected to the database from a terminal. The user can simply take the necessary data with them in their iPhone.

So how did we accomplish this? Well, we used a synchronization process. The first time the user uses the application, they need to have access to the database. Once they have connected to the database at least once, the information from the desired table will be stored in the phone. After this, the user can take the phone wherever they desire and they will be able to edit the local copy of the data irrespective of whether they are connected to the Informix database or not. The next time they are connected to the database, they can synchronize the information from their phone into the database.

What is needed from the developer?

The application developer must first have knowledge of some very basic but important concepts. First of all, and this might be obvious, the developer must know how to develop in the iPhone SDK environment and a basic understanding of Objective-C.

Another important concept the developer must know of is Core Data. Core Data is a pretty extensive topic and it requires some research and time to understand it thoroughly. In simple terms, what Core Data does is that it provides the developer with a way to communicate with the SQLite data on the phone without the need for SQL statements. The developer can programmatically access the data using Objective-C statements. In order to accomplish this, the developer should understand concepts like creating a managedObjectModel and managedObjectContext.

What is needed from the end user?

In order to use the demo app itself, the user must know which table(s) they actually want to work with since at the beginning of the application they will be given the option to select the table they desire.

Apart from that, what is really required from the user is to know which column is the unique key for that specific table. This is required because the application assumes there is a unique key and it works around that not only to organize the details, but also to make sure there aren’t duplicates during synchronization and updates. In the end, the unique key is really important to make the application work.

How do we connect to the database?

In order to connect to the database we use a generic web service (two to be exact) written in PHP. Through this web service we create an XML file that gets parsed in order to get the desired information. For parsing, we use the open source parsing code from TBXML Source Code.

The only thing the developer has to change from the web service is the connection information so they can connect to their own database. The developer can also make the web service more sophisticated if they find a need for it in order to solve some of the limitations the app currently has. We will talk more about this limitations in the near future.

<?php
# Input SQL stmt - need to receive from the app via POST
$sql = $_POST['sql_stmt'];
$table = $_POST['tableName'];

# setup some variables for our connection.
# change the following for your environment.
$host = HOSTNAME; #Hostname of where Informix is running.
$port = PORTNUM;  # port # that Informix is listening on.
$server = SERVERNAME;  # The INFORMIXSERVER.
$database = DATABASENAME; # Name of the stores_demo database.
$protocol = PROTOCOLNUM;  # Protocol for the connection.
$user = USERNAME;  # Username to connect as.
$password = PASSWORD; # Password for the user.

#setup our ‘dsn’
$dsn  = "informix:host={$host};service={$port}";
$dsn .= ";database={$database};protocol={$protocol}";
$dsn .= ";server={$server}";

# connect to the database.
$db = new PDO($dsn,$user,$password);
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

# perform the query.
#$count = $db->exec($sql); #this is what we use when we didn't deal with error

# perform the query and if there is an error, return it.
try {
	$count = $db->exec($sql);
	}
catch (PDOException $e) {
	$thereWasAnEror = 'YES';
		echo $thereWasAnError;

		#create a simplexml.
		$xml = simplexml_load_string("");

		$errorMessage = 'Error: ' . $e->getMessage();
		$errorMessage = strrev($errorMessage);

                #we are deleting the last parenthesis which contains unecessary info.
		$errorMessage = preg_replace("/\)[^\)]+\(/","",$errorMessage, 1);

		$errorMessage = strrev($errorMessage);

		$node = $xml->addChild("$table");
		$node->addChild('ThereWasAnError', 'yes');
		$node->addChild('ErrorMessage',"$errorMessage");

		#display the xml;
		print $xml->asXML();

		exit();
	}

# create the select statement to verify if the data was inserted
$sql = "SELECT * FROM $table";

# perform the query
$stmt = $db->query($sql);

# fetch all the data.
$rows = $stmt->fetchAll(PDO::FETCH_ASSOC);

#close the query.
$stmt->closeCursor();

# we are going to output this in an xml format so
# lets tell the browser to expect xml
header("Content-Type: text/xml");

#create a simplexml.
$xml = simplexml_load_string("");

# loop thru our records adding each one to our xml.
foreach ($rows as $key => $value)
{
        $node = $xml->addChild("$table");
        $node->addChild('ThereWasAnError', 'no');

        foreach ($value as $k => $v)
        {
                #fix up any special characters.
                $v = htmlentities($v);
                $node->addChild($k,$v);
        }
}

#display the xml;
print $xml->asXML();

?>

Note: you will notice from the source code itself, that we use two generic web services. The second one is just like this one, with some changes. The second one contains an if statement, to make sure we have a tableName (even if we just call it tableName) and we deleted the Select * From statement. This is because we use the second generic web service to gather the column names as well as the table names (which we gather from the catalog’s table) and if we have the Select * From the way it is in the first generic web service, then the Select would override the SQL Statement to receive the column names and table names.

Also, the second web service doesn’t deal with error messages, so we didn’t include that in this one. 

Here is the second generic web service: 


<?php
# Input SQL stmt - need to receive from the app via POST
$sql = $_POST['sql_stmt'];
$table = $_POST['tableName'];

if($table == NULL)
{
$table = 'tablename';
}
# setup some variables for our connection.
# change the following for your environment.
$host = HOSTNAME; #Hostname of where Informix is running.
$port = PORTNUM;  # port # that Informix is listening on.
$server = SERVERNAME;  # The INFORMIXSERVER.
$database = DATABASENAME; # Name of the stores_demo database.
$protocol = PROTOCOLNUM;  # Protocol for the connection.
$user = USERNAME;  # Username to connect as.
$password = PASSWORD; # Password for the user.

#setup our ‘dsn’
$dsn  = "informix:host={$host};service={$port}";
$dsn .= ";database={$database};protocol={$protocol}";
$dsn .= ";server={$server}";

# connect to the database.
$db = new PDO($dsn,$user,$password);

# perform the query.
$count = $db->exec($sql);
# perform the query
 $stmt = $db->query($sql);

# fetch all the data.
 $rows = $stmt->fetchAll(PDO::FETCH_ASSOC);

#close the query.
 $stmt->closeCursor();

# we are going to output this in an xml format so
 # lets tell the browser to expect xml
 header("Content-Type: text/xml");

#create a simplexml.
 $xml = simplexml_load_string("");

# loop thru our records adding each one to our xml.
 foreach ($rows as $key => $value)
 {
 $node = $xml->addChild("$table");
 foreach ($value as $k => $v)
 {
 #fix up any special characters.
 $v = htmlentities($v);
 $node->addChild($k,$v);
 }
 }

#display the xml;
 print $xml->asXML();

?>

The web service receives the SQL statement as a parameter as well as the table name. This SQL Statement is then passed onto the Informix Database and then the results is passed back in an XML file and then the application parses through this XML file to get the information.

There are cases where there is nothing returned in the XML file, cases like inserting or deleting or updating for example, don’t return anything. In that case, the SQL statement is simply passed into the Informix Database for it to perform the desired action.

In the first web service we also have a try-catch that deals with the error message. If there is an error message returned by the server, the web service presents it in an XML file that we later use to parse through, and grab the error message, and then the web service exits. We then use this error message and display it in the application for the user to see what happened.

How synchronization works

Both the user and the developer must understand precisely how the synchronization is being done. When the application first launches, the information is gathered from Informix and stored into Core Data. We only grab new rows in this process though, any changes to the current rows are not considered. After that synchronization is done, we then check if any of the information within Core Data was updated and if it was, we synchronize it into Informix. All of this is done automatically when the application launches. Now, because of the order, the user needs to keep one thing in mind: if they deleted a row from Core Data that hasn’t yet been synchronized into Informix (meaning that it hasn’t yet been deleted, be it because there was no connection at the time or something along those lines) then that row is going to be added into Core Data again once this synchronization is done because the application is going to assume that it is a new row.

While the application is running, the user has the ability to manually sync from Informix to Core Data and vise versa. Again, from Informix to Core Data we only insert new rows, we don’t deal with updates or deletes. Whereas if we sync from Core Data to Informix, we deal with everything (insert, delete and update). The assumption here is that the data in the phone is always the most current data.

At the end, when the application closes, the application will sync everything that is new in Core Data into Informix. Note: if there is no connection at the time, any deleted rows in Core Data will be inserted again once the application is launched again. They weren’t deleted from Informix since there was no connection at the time of closing.

September 12, 2011


Pages

Categories

Archives