] > Pau Mendoza DISCA, Universidad Politecnica de Valencia

e-mail: pabmench@disca.upv.es

Patricia Balbastre DISCA, Universidad Politecnica de Valencia

e-mail: patricia@disca.upv.es

January 2002 2002 OCERA Consortium

Name Native CBS implementation based in RTLinux. Description This component implements the CBS scheduling policy for RTLinux threads. The Constant Bandwidth Server (CBS) was developed to efficiently handle soft real-time requests with a variable or unknown execution behaviour under EDF scheduling policy. Authors Pau Mendoza and Patricia Balbastre. Reviewer Alfons Crespo. Layer Low level RTLinux. Version 0.1 Status Stable Dependencies EDF scheduler for RTLinux. Release Date M2

The problem of integrating flexible Quality of Service (QoS) guarantees in real-time systems has been widely studied in the last years, resulting in some interesting proposals. Probably, the most important theoretical result that emerged in this work is the idea that in order to provide a predictable QoS to different applications running on the same system, the OS kernel must provide temporal isolation between different applications or tasks. Temporal isolation (also known as temporal protection), requires that the temporal behaviour of a task is not influenced by the temporal behaviour of other tasks in the system. Based on classical real-time scheduling (EDF or RM priority assignment), it is possible to implement a reservation guarantee by simply enabling a task to execute as a real-time task (scheduled, for example, by EDF or RM) for the reserved time Q, and then blocking it (or scheduling it in background as a non real-time task) until the next reservation period. One of the most interesting resource reservation protocols is the Constant Bandwidth Server (CBS). The Constant Bandwidth Server (CBS) was developed to efficiently handle soft real-time requests with a variable or unknown execution behaviour under EDF scheduling policy. To avoid unpredictable delays on hard real-time tasks, soft tasks are isolated through a bandwidth reservation mechanism, according to wich each soft task is assigned a fraction of the CPU and it is scheduled in such a way that it will never demand more than its reserved bandwidth, independently of its actual requests. This is achieved by assigning each soft task a deadline, computed as a function of the reserved bandwidth and its actual requests. If a task requires to execute more than its expected computation time, its deadline is postponed so that its reserved bandwidth is not exceeded. As a consequence, overruns ocurring on a served task will only delay that task, without compromising the bandwidth assigned to other tasks. By isolating the effects of task overloads, hard tasks can be guaranteed using classical schedulability analysis. In more detail, as stated in CBS can be defined as follows: A CBS server is characterized by a budget cs and by the server bandwidth Us=Qs/T s. Where Qs is the maximum budget and Ts is the period of the server. Initially, the server is assigned the deadline ds=0 Each served job, is assigned the deadline of the server. Whenever the server task executes, the budget cs is decreased by the same amount. When cs=0 the server budget is recharged to the maximum value Cs and a new server deadline is generated as ds,k+1=ds,k+Ts. When a job Ji,j arrives and the server is idle (there are no pending jobs), if cs ≥ (ds, k-ri,j)Us the server generates a new deadline ds,k+1=ri,j+ Ts and cs is recharged to the maximum value Qs When a job finishes, the next pending job is served using the current budget and deadline. In the CBS algorithm is extended to deal with shared resources, specifically, it is integrated with the Stack Resource Policy (SRP) . This component implemetns the CBS scheduling policy for RTLinux threads, that is, when the CBS thread is the highest priority one, it can execute Linux task. How linux processes are scheduled is not related with this component.

This is a low-level RTLinux and low-level Linux component, since it modifies both RTLinux and Linux kernel. Thus, the distribution of this component comes in the form of two patches, one for RTLinux (rtlinux-3.2pre1-rtlcbs.patch) and the other for Linux (linux-2.4.18-rtlcbs.patch).

To initialise CBS threads, new API functions have been defined. Specifically: extern inline int pthread_attr_set_initbudget_np pthread_t thread hrtime_t initbudget extern inline int pthread_attr_get_initbudget_np const pthread_t thread hrtime_t *initbudget extern inline int pthread_setinitbudget_np pthread_t thread hrtime_t initbudget extern inline int pthread_getinitbudget_np pthread_t thread hrtime_t *initbudget extern inline int pthread_initcbs_np pthread_t thread hrtime_t period void make_linux_task_cbs_server hrtime_t start hrtime_t deadline hrtime_t period int priority

CBS depends on EDF scheduler, since aperiodic events are scheduled with EDF scheduling policy. V1 API support has not to be selected when configuring RTLinux installation.

This is a stable version. Some bugs have been already fixed from the beta version.

The implementation work consists of modifications to two files: rtl_sched.h and rtl_sched.c. The modifications are mainly in the scheduler function rtl_schedule(), apart from the new functions and parameters added. A new scheduling policy is defined (SCHED_CBS_NP), for those threads working as CBS servers. All the modifications done to the RTLinux code have been enclosed by macros (CONFIG_OC_RTLCBS) that can be toggled from the RTLinux configuration tool. Each CBS server thread must have two parameters: The maximum budget Qs. It has been added to the rtl_sched_param struct, because usually params belonging to this struct are initialized once, and after this, they do not change its value. The parameter has been called sched_cbs_init_budget: struct rtl_sched_param { int sched_priority; hrtime_t sched_deadline; #ifdef CONFIG_RTL_CBS hrtime_t sched_cbs_init_budget; #define RTL_INIT_BUDGET(th) ((th)->sched_param.sched_cbs_init_budget) #endif }; The current budget cs. This param decreases and it is recharged as time goes by. So, it is more suitable to include it in the rtl_thread_struct struct: struct rtl_thread_struct { ... fields of original RTLinux ... #ifdef CONFIG_RTL_EDF hrtime_t current_deadline; int policy; #ifdef CONFIG_RTL_CBS hrtime_t sched_cbs_current_budget; #endif #endif }; When an aperiodic event arrives, depending on wether the CBS thread is idle or not, a new deadline for the CBS thread is calculated. This deadline is assigned to the thread associated with the aperiodic event. The implementation has been done in such a way that the user can choose between two options: The aperiodic events are interrupts, and the thread associated with these events is the linux task. When an interrupts arrives (for example, a network interrupt), linux executes until it schedules the idle task. In this implementation, there is no need to implement queues of pending jobs. As an example, let's suppose that two interrupts arrive at the same time. In the first interrupt arrival, if the CBS server is idle, a new deadline is calculated for the linux task. Linux task will recover its original state (as the background thread) when it schedules the idle task. So, the second interrupt will be already treated. If not, linux do not schedule the idle task until all tasks are finished. The user can customize CBS. Therefore, events have to be defined by the user, and the same CBS thread attends the execution of events. When the event processing finishes, the CBS thread suspends itself. In this option, the user must implement the queues of pending jobs. From now on, the CBS thread will refer to the linux task or the CBS thread itself, depending on the option chosen. In the first option, when a network interrupt arrives, the linux task must execute with the priority (static and dynamic) of the CBS server thread associated with the interrupt. This way of inheritance between the linux task and the CBS thread is made in the interrupt handler, through the make_linux_task_cbs_server function. In this function, the CBS thread parameters (initbudget, deadline, period and priority) are attached to the linux task. Then, a new deadline is calculated in the rtl_cbs_reset_deadline function. The code of both functions is listed below: policy != SCHED_CBS_NP) { sched = &rtl_sched[cpu_id]; now = sched->clock->gethrtime(sched->clock); RTL_INIT_BUDGET(linux_task) = initbudget; linux_task->sched_param.sched_deadline = deadline; linux_task->period = period; linux_task->sched_param.sched_priority = priority; linux_task->policy = SCHED_CBS_NP; linux_task->sched_cbs_current_budget = 0; if (linux_task->current_deadline < now) linux_task->current_deadline = now; cbs_linux_idle = 1; rtl_cbs_reset_deadline(linux_task); rtl_reschedule_thread(linux_task); }; return; }]]> policy == SCHED_CBS_NP)) { now = sched->clock->gethrtime(sched->clock); if (now + (t->sched_cbs_current_budget/ RTL_INIT_BUDGET(t)) * t->period >= t->current_deadline) { t->current_deadline = now + t->period; t->sched_cbs_current_budget = RTL_INIT_BUDGET(t); } t->resume_time = now; } rtl_restore_interrupts(flags); }]]> If the second option is chosen, there is no need to use the make_linux_task_cbs_server function, since the CBS thread already has the correct parameters. Simply, the rtl_cbs_reset_deadline function is executed when the CBS thread is woke up. This is made in the pthread_kill function: Then, the CBS thread will execute when its new deadline is the nearest of all active threads. It is important to note that if another interrupt arrives, the rtl_cbs_reset_deadline function will not execute, since the thread is already active. By definition of CBS, if a job arrives and the server is active the request is enqueued, so it will execute with the current deadline. When the CBS thread is chosen by the scheduler as the highest priority thread, two situations can stop the thread execution: A higher priority thread becomes active in the future, but before the CBS thread finishes its execution. This is solved in the find_preemptor function in the original RTLinux code. The budget is exhausted, so it must be recharged and a new deadline is assigned. In the original RTLinux code, the last situation is not taken into account, so if the thread is not preempted, the timer is programmed to ten miliseconds after. Now, the timer must be programmed to the nearest time of the previous cases commented: a higher thread activation, or the actual time plus the current budget. The code is: clock->mode == RTL_CLOCK_MODE_ONESHOT && !test_bit (RTL_SCHED_TIMER_OK, &sched->sched_flags)) || (new_task->policy == SCHED_CBS_NP)) { if ( (preemptor = find_preemptor(sched,new_task))) { #ifdef CONFIG_OC_RTLCBS if (new_task->policy == SCHED_CBS_NP) { if (( preemptor->resume_time - now) > (new_task->sched_cbs_current_budget)){ (sched->clock)->settimer(sched->clock, new_task->sched_cbs_current_budget); #endif (sched->clock)->settimer(sched->clock, preemptor->resume_time - now); #ifdef CONFIG_OC_RTLCBS } } else { (sched->clock)->settimer(sched->clock, preemptor->resume_time - now); } #endif } else { #ifdef CONFIG_OC_RTLCBS if (new_task->policy == SCHED_CBS_NP) { (sched->clock)->settimer(sched->clock, new_task->sched_cbs_current_budget); } else #endif (sched->clock)->settimer(sched->clock, (HRTICKS_PER_SEC / HZ) / 2); } set_bit (RTL_SCHED_TIMER_OK, &sched->sched_flags); }]]> Once the CBS thread is preempted, the scheduler must measure the execution time, in order to decrease the current budget. When the budget is exhausted, it must be recharged and a new deadline is generated, according to CBS rules. This must be checked in the rtl_schedule() function before choosing the new task to execute. However, if the linux threads executes as the CBS server (first option) and it schedules the idle task (which is indicated by the flag cbs_linux_idle) then it must recover its original state, that is, as the background thread. rtl_current->policy == SCHED_CBS_NP) { if (sched->rtl_current->sched_cbs_current_budget >= elapsed_time){ sched->rtl_current->sched_cbs_current_budget -= elapsed_time; } else { sched->rtl_current->sched_cbs_current_budget = RTL_INIT_BUDGET(sched->rtl_current); sched->rtl_current->current_deadline += sched->rtl_current->period; sched->rtl_current->resume_time = now; } if (sched->rtl_current == &sched->rtl_linux_task) { if (cbs_linux_idle == 0) { sched->rtl_linux_task.sched_param.sched_priority = -1; #ifdef CONFIG_RTL_EDF sched->rtl_linux_task.sched_param.sched_deadline = 0; sched->rtl_linux_task.current_deadline = 0; #endif sched->rtl_linux_task.sched_cbs_current_budget = 0; RTL_INIT_BUDGET(&sched->rtl_linux_task) = 0; sched->rtl_linux_task.period = HRTIME_INFINITY; sched->rtl_linux_task.policy = SCHED_OTHER; } } }]]> The only point to solve now is to detect wether the linux task schedules the idle task. To do this, the linux code must be modified, specifically the kernel/sched.c, which is where the linux scheduler is implemented. The flag cbs_linux_idle is defined as extern in the include/sched.h file. It has the value 0 whenever the idle task is scheduled, and 1 otherwise. Then, simply, before the context switch in the schedule() function, it is added the following code:

The CBS implementation in RTLinux must have the following behavior: If the served tasks have the same period as the CBS server, then the CBS algorithm behaves as plain EDF. The CBS server must reclaim any spare time caused by early completions. Regarding context switches, the CBS server must call the scheduler at most every budget units of time. Therefore, no more of (task computation/server budget) context switches must be introduced by the new implementation of CBS. Also, a low overhead have been assured, since there is no need to implement queues of pending aperiodic arrivals.

Four tests have been implemented to validate the correct behaviour of the system. These tests can also be used as examples for the user, since it creates CBS threads to validate the correct behavior. The target system was a Pentium 133 MHz. For every test developed, it has been used the same workload, that is, threads scheduled under EDF are the same in all tests. The parameters of the EDF periodic threads are listed in . CBS parameters will be presented when explaining the specific tests. Task id Compute Deadline Period Prio level T1 0.8 6 6 10 T2 2.4 10 10 10 T3 3 11 11 10 T4 3.5 19 19 10 The structure of the tests is made up, basically, of two files: rt_process.c and CBS_app.c. The former contains the module that creates the four threads plus the CBS threads. The latter contains the code that generates the aperiodic events. The code to implement the four periodic threads is listed below: While implementing these tests, some debugging code was also included. That feature was used to obtain the execution chronogram of the threads for each test. The debugging can be made selecting the option CONFIG_RTL_EDF_DEBUG in the config menu of RTLinux install. This way, it is obtained a trace via the rtf0 where the scheduler writes the thread identifier, initial and final execution times and activations of the thread, in a text file. This file is the input data to a X window application "crono" that represents the chronogram of the execution.

This test has five threads: the four threads shown in , plus the CBS thread. Parameters for the CBS thread are presented in Task id Compute Deadline Period Prio level Max. budget CBS1 1.2 3 3.1 10 0.15 The code implemented to create the CBS thread is the following: This thread also executes the code of aperiodic events. These events are generated by a user application by writing on a rt_fifo. Following, is the code of the user application (CBS_app): As it shows the previous code, the user application generates two aperiodic events. The first event wakes up the CBS server (by means of the rt_fifo handler), that executes a dummy loop to consume its computation time. Afterwards, the thread suspends itself until the arrival of the second event. The code that executes the CBS server is: Note that the CBS thread is not make periodic, that is, the new function pthread_initcbs_np is used instead of pthread_make_periodic_np. If the latter function is used, the CBS thread would be woke up in every period. This behaviour is not correct for a CBS server, since it must be woke up only when an aperiodic job arrives. Following is the chronogram () of the execution:

The chronogram generated by the execution shows the four periodic tasks plus the CBS server (task 5 in the chronogram) and the linux thread (task 0). In the first three replenishments CBS executes without interferences (since its deadline is shorter than others), but in the fourth it is preempted by task 1, and later by task 2. This is because in each replenishment the deadline is increased by the period.

It is possible that when the CBS thread wakes up and executes, no other threads are active. In this situation, several replenishments can occur before other threads become active. Therefore, CBS deadline becomes greater, and at the end it behaves as the background thread. To increase the system utilization, this test is made up of the four periodic threads in and two CBS threads. This way, CBS and periodic threads will alternate their executions whenever a replenishment occurs. Parameters for the CBS threads are presented in . Task id Compute Deadline Period Prio level Max. budget CBS1 0.6 3 3 10 0.2 CBS2 0.84 5 6 10 0.3 Regardin the implementation details, an periodic auxiliary thread is created to wake up CBS threads when the user application writes on a rt_fifo. This auxiliary thread wakes up all CBS threads, so they compete for the processor control. The code is following: The chronogram is shown in . Again, tasks 1 to 4 are the periodic threads, tasks 5 and 6 are the CBS ones and task 7 is the auxiliary thread. The figure shows how the CBS thread with the shorter deadline wakes up and executes after the execution of the auxiliary thread. In its first replenishment, its deadline becomes greater than the other CBS thread, so CBS2 executes. CBS1 and CBS2 executions are alternating until periodic threads with shorter deadlines become active.

Tests 3 and 4 shows how to associate linux task to serve aperiodic events. Is is considered that the aperiodic job has been served when linux schedules the idle task, and new events arrive when an interrupt is forwarded to linux. In test 3, aperiodic events are generated as in the previous test, that is, by means of the user aplication that writes on a fifo. The fifo handler now do not wake up the CBS server, since in this test is linux who serves events. Instead, linux executes as a CBS thread with its properties. This is made in the make_linux_task_cbs_server function: The parameters of the linux task when executing as CBS server are shown in . Task id Deadline Period Prio level Max. budget Linux CBS 3 3 10 0.2 The chronogram is presented in . The chronogram generated by the execution draws the Linux task in a different position depending on wether it is scheduled as the task in background (task 0), or as a CBS thread (task 5).

This test is similar to the previous one, it only changes the way of detecting aperiodic events. Now, aperiodic events are parametrized interrupts. Therefore, there is no user application file (CBS_app). By default, the interrupt detected is the keyboard, but network interrupts or other kind of interrupts can be easily detected, with small modifications of the code. When an keyboard interrupt occurs, the interrupt handler attach CBS properties to the linux task by means of the make_linux_task_cbs_server function: The parameters of the linux task when executing as CBS server are shown in . Task id Deadline Period Prio level Max. budget Linux CBS 3 3 10 0.5 The chronogram is presented in . The chronogram generated by the execution draws the Linux task in different position depending on wether it is scheduled as the task in background (task 0), or as a CBS thread (task 5).

To install the cbs scheduling support in rtlinux, you should: Install the rtlcbs ocera component in your source code: Change directory to the component main one: 'cd rtlcbs_directory' Edit the file 'Makefile' and set your rtlinux and linux source directory, for example: "RTLINUX = /usr/src/rtlinux-3.2-pre1" "LINUX = /usr/src/linux-2.4.18" Type make install in the main directory of the rtlcbs component. This will install the component: Copying the documentation, the examples. If necessary, (if it is not installed yet) will install the rtledf features patching your rtlinux with a retailed version. And will patch your source code to support the cbs scheduling. After this, you must rebuild your kernel image and your RTLinux modules as usual: Rebuilding the kernel image: cd /usr/src/linux-2.4.18 make clean dep make {your_image_type} Install it (i.e. with lilo) Rebulding your RTLinux: cd /usr/src/rtlinux-3.2-pre1 make clean make make modules_install And finally, reboot your system using the new kernel.

&biblio;