Moving the Blog

I will discontinue this blog on blogger and move to this one instead.

Unicode in ILIAS - Sorting things alphabetically

There are several places in ILIAS where titles, terms or other text items are sorted alphabetically. In many cases the ordering is done by the database, in other cases by PHP. Since ILIAS works with UTF-8 encoded text, ordering is not a trivial task. MySQL offers support for UTF-8 since version 4.1. PHP will offer full unicode support with version 6.

Before that the standard PHP string functions cannot handle UTF-8 properly. ILIAS uses the PHP multibyte string extension if activated. This solves already a lot of problems, e.g. determination of string lengths or substrings. But this extension does not include a function to compare two strings and get their correct alphabetical order.

So we used a function called strcoll for that. This function uses the current locale setting to determine the ordering of strings. In ILIAS (>= 4.0) you set the locale in the administration under "General Settings". This seems to work on many linux based systems and on OS X sometimes. But unfortunately not always.

It worked under Leopard for me, but with Snow Leopard I suddenly run into problems. I am not sure what changes have been responsible for this exactly, but I found a PHP bug report that explains, that the PHP function uses the underlying system strcoll function - and this one is broken sometimes.

There is another extension of PHP: The internationalization extension. This one comes as a PECL extension for PHP < 5.3 and is included (but must be activated when configured) in PHP 5.3. I was not able to get it running under Snow Leopard, but on Ubuntu 9.10 you need to:

• Install package libicu-dev
• Run > pecl install intl (with directory "/usr" when asked)

If pecl complains about the pre-processor you may also install package "build-essential".

With version 4.0.2 ILIAS detects whether the intl extension is activated and uses its functions to compare strings with the locale setting in the administration. And finally, everything should be in the correct order.

Comparing ILIAS 3.10 and 4.0 Performance

I tried to compare the performance of ILIAS 3.10 and 4.0 in a major scenario, the presentation of a course. Since I had no productive environment to test on, I used an "artificial" setup containing

• 1000 Users
• 200 Categories
• 300 Courses
• 40 Files per Course (12.000 in total)
  
• Activated Calendar and News
• 30 Calendar Appointments per Cours (9.000 in total)

We have a little server not optimized for performance in any way. Database and web server on the same machine. Since I did not want to test the hardware I didn't put the server "under stress". I only simulated 10 simultanous user doing requests every 5 to 15 seconds using jMeter. Besides the request to the main course page, they accessed categories and the personal desktop, too. But here I only examine the requests to the course page. I deactivated all asynchronous loading of information, so all information is gathered in one request. Here is how the course looks like:



The following figure shows the results for 3.10 and 4.0 with different activated caches/settings. Every user accessed two different courses three times. Different users accessed separated courses. All in all there have been 10 * 2 * 3 = 60 course requests. So each bar represents 20 requests: The first bar allways the first request to a course of a user, the second and third bar correspond to the second and third request of a user to the same course. These two bars show caching effects (also the first does, but in a negative way).


The first group of bars labeld "3.10" shows the result for ILIAS 3.10.10.

The second group "4.0" is ILIAS 4.0.1 without any special settings. This is about 5% faster, which is not much, but at least it's not slower than 3.10. ILIAS 4.0 does more in this time, since it displays all actions (even if hidden) and has to get permission information, e.g. for "Delete" actions, which are not shown per default in 3.10 ("All Commands" deactivated). Also the rendering of the action lists (drop down) in 4.0 is more complex and we expected the database abstraction layer to cost additional time. But we also made a number of little performance improvements in 4.0 which seem to have a positive effect here.

The third group "4.0 NC" has the news caching activated. The effect is not very big. And this even after the news cache in 4.0 has been revised and should work better than the one of 3.10 (only available on the personal desktop). The effect may be this low, since the majority of request time is spended with displaying the 40 file items on the screen and their actions. I expect a much bigger effect, if a large number of files are stored in subfolders and only a small number of folders needs to be displayed on top.

The group labeld "4.0 NC RC" has an additionally activated repository cache (Administration -> General Settings). Since the cache stores permission information per user and item to the DB you have a significant negative effect with the first request to a course (or any other repository page). The cache is only used on this level, not when you navigate into an item (e.g. clicking on "Edit" for a file) - here permissions information will always be "up-to-date". The second and third request to the course show a big positive effect, since the cache replaces a lot of database requests. However it will be interesting to see the results of this "mixture" of positive and negative effects in a productive scenario. In our scenario the average time over all three requests was more than 10% faster.

The last group combines news cache, repository cache and a setting that is called "Load Resource Action Lists Asynchronously" (Administration -> General Settings). Since the new visualization of the action lists does not show any action items before the users clicks on "Actions", we can load this information asynchronously. This results is a big reduction of database accesses. The only disadvantage is: There is currently no fallback, if Javascript is deactivated. You will not be able to use the actions in this case, they will be "invisible".

Again, please keep in mind that all this is one scenario in an artificial setup. Every ILIAS installation shows different behaviour depending on hardware, configration and usage. Please add your findings into our performance wiki.

PHP Profiling

In ILIAS we used a separate Benchmark class for performance measurements in the past. It used PHP's microtime() to calculate processing time and store it to DB. The class is quite old and there are a some disadvantages. Developers have to add extra code to make measurements and if an administrator activates it on a productive system he may be in real trouble because it slows down the system significantly.

There is a better solution: xdebug and kcachegrind (for kde/linux) / wincachegrind(windows). I am using xdebug since a long time, but I didn't try to make kcachegrind run under OS X yet. I should have done earlier. Stefan recommended it to me this week and I found that there was a simple way to install it using MacPorts. The basic steps are (Snow Leopard's PHP 5.3):

> sudo pecl install xdebug

php.ini:
extension=xdebug.so
xdebug.profiler_enable=1 ; 0 deactivates the profiler
xdebug.profiler_output_dir=/tmp
xdebug.profiler_enable_trigger=1
xdebug.profiler_output_name=cachegrind.out%p

Install MacPorts.

> sudo port install kcachegrind
> sudo port install graphviz
> sudo kcachegrind

You will need to have XCode installed as prerequisite and some disk space and time, since there are a lot of dependencies for kcachegrind.


This is a screenshot of an Xdebug output visualized by kcachegrind. I usually walk through the list on the left from top to bottom. The list starts with entries like "main" that cover the whole request. The visualization map on the right looks very complex for these entries and doesn't give much information. When moving downwards some entries will appear where larger areas of the same color (equals functions that need more time) appear on the right. If there is no good reason why a function should need this much time, you should start to investigate the code.

Load Testing using jMeter

This week we have added a new service to ILIAS called "LoadTest" (found in directory Services/LoadTest). It sets up a very basic framework for load tests in ILIAS using jMeter. JMeter provides a Java desktop application for configuring and running tests, but can be managed from the command line, too.


The screenshot shows an example load test configuration in jMeter. The structure is outlined on the left side. Main parts are request items that call different screens in ILIAS. The example includes a login request and two sets of category/course/personal desktop requests. On the right side you see the detailed view of a category request. JMeter stores this information in XML files with the suffix ".jmx".

The ILIAS service includes one class "ilDataLoader" that can be used to generate an ILIAS database with certain numbers of objects that should be used during the tests. It provides functions for:

• Loading/storing DB templates
• Generating users, categories, courses, files (in courses) and calendar entries
• Generating CSV files containing all course/category/user IDs. These files will be used by jMeter, e.g. to login the users or to create requests to a number of categories/courses.

The service also includes a CLI PHP script called run_loadtest that can run multiple load test by calling jMeter and storing its results. The scripts reads a configuration file that defines:

• Pathes of jmeter, ILIAS web directory, test files (jmx), result files and input CSV files
  
• ILIAS client to be used
• Number of users, test loops and "ramp-up" time

Finally you will find a jmx directory in the service that contains the jMeter load test files (currently only the one above).

All this is still very sketchy. I did not start a documention on it in our developer guide yet since I am sure that many things will change soon. But everyone is invited to checkout the latest trunk and try things out.

Outlook: Upgrading to ILIAS 4.0

First for those of you who remember the upgrading procedure from ILIAS 2 to ILIAS 3: It will not be like this. There was no automatic database update procedure from ILIAS 2 to ILIAS 3. Most data had to be exported from ILIAS 2 and re-imported into ILIAS 3.

ILIAS 3 was a complete reprogramming. Not one line of code or database table of ILIAS 2 was re-used. Everything had been setup from scratch.

The upgrade to ILIAS 4 will basically work like every ILIAS 3.x upgrade. The ILIAS Setup will take care of the database migration. What will be different?

• You will need to upgrade to ILIAS 3.10 before performing the step to ILIAS 4.0, if you are using ILIAS 3.9 or lower.
  
• The setup includes a new option "Next Break" that allows you to perform a sequence of smaller sets of database update steps. The complete database update from 3.10 to 4.0 will contain more than 1000 steps, so you can split this into a sequence of runs of e.g. around 200 steps.

Splitting up the database update allows to prevent problems with execution time limits or memory consumption. If you experience a severe problem with one particular database update step you may even run it up to the step before, perform a database backup and focus on this one step.

IDE and Code Templates

I switched from jEdit to the Aptana PHP IDE recently (Stefan has recommended it to me) to write my ILIAS code. In the past, at least one time a year I tested some PHP IDE for eclipse. The main reason why I stayed with jEdit was performance and stability (I am working on MacOS). Aptana/Ecplipse is still slower than jEdit, but I have been convinced now. It is stable and offers a lot of nice features, e.g I like they way code templates are organised.

Code templates are very important to speed up your development. I have uploaded my current code templates file for Aptana to the ILIAS Development Guide. If you like to share yours, please send it to me.

Update 18 Oct 2009: The Aptana PHP Plugin will be discontinued. There will be a joint effort with the PDT team.

Repository Plugin Slot in ILIAS 4.0 (formerly known as ILIAS 3.11)

The upcoming ILIAS 4.0 will offer a plugin slot for new repository resources. This enables you to implement new types of learning resources without the need to make it a part of the ILIAS core. All you need is a set of PHP classes. The full documentation can be found here.

The slot will be considered as beta, which means that you may expect some changes in the interface during bugfix releases, but we will try to avoid this. However I am sure that there is a lot to improve. So please share your experience with us if you create any new components.