Support forums : You can sense the bug

Scheduled hitlog purging (cron)

Not like "it smells funny when I click that". This is for bugs that don't show an error but something just doesn't seem right or ends up doing something you think is wrong. It doesn't show you an error but you know something ain't right.

Moderator: Dracones

Scheduled hitlog purging (cron)

Postby BushLeagueCritic » Thu Nov 18, 2010 6:33 pm

There's a bug somewhere that I can't track down. After the "number of days worth of hits" to keep has been reached, it starts trying to run older completed tasks... resulting in failures and multiple pending purge jobs for the next day. It blows up exponentially from there.

After clearing both cron tables (log and task) here is what I get (I have my preferences set to keep 7 days of hit data):
Days 1-7 -- everything behaves normally. At the end of seven days, I have eight entries (id's 1 - 8) in the task list... the seven previously successful jobs plus the job pending for the next day.

Day 8 -- Job (id 8) runs successfully and reschedules (as job id 9) for the next day. Then it (for whatever reason) tries to execute a previously completed task (job id 1) which fails and then reschedules as job (id 10). The result is two pending jobs for the next day.

Day 9 -- Job (id 9) runs successfully and reschedules. Job (id 10) fails and reschedules. Then jobs (id 1 & 2) - again for unknown reasons -- attempt to execute... fail and reschedule. Resulting in four scheduled task for the next day.

Lather, rinse, repeat until the task list is full of failed and spurious pending jobs.

I can't seem to find the code that determines which tasks to execute (nor the code that resubmits a repeating task as a new task).

Help!
BushLeagueCritic
User avatar
 
Posts: 47
Joined: Wed Jul 21, 2010 2:50 pm

Re: Scheduled hitlog purging (cron)

Postby Yabs » Sun Nov 21, 2010 3:24 pm

That sounds like a fun one to track down.

¥
I may have opened the door but you entered of your own free will

Image Techno Babble II
Image Tacky Pad 3
Yabs
Dracone
User avatar
 
Posts: 896
Joined: Sat Nov 21, 2009 9:59 am

Re: Scheduled hitlog purging (cron)

Postby Yabs » Sun Nov 21, 2010 3:46 pm

Ooooooook, that just made my head hurt :-S Can you do a dump of your cron_log && cron_task tables and attach them?

¥
I may have opened the door but you entered of your own free will

Image Techno Babble II
Image Tacky Pad 3
Yabs
Dracone
User avatar
 
Posts: 896
Joined: Sat Nov 21, 2009 9:59 am

Re: Scheduled hitlog purging (cron)

Postby BushLeagueCritic » Sun Nov 21, 2010 4:16 pm

Ooooooook, that just made my head hurt

Tell me about it. My head's been hurting for a week! :lol:
Attachments
qp_cron__log.sql.gz
(835 Bytes) Downloaded 447 times
qp_cron__task.sql.gz
(829 Bytes) Downloaded 434 times
BushLeagueCritic
User avatar
 
Posts: 47
Joined: Wed Jul 21, 2010 2:50 pm

Re: Scheduled hitlog purging (cron)

Postby Yabs » Sun Nov 21, 2010 4:27 pm

You have my sympathy ;) Thanks for the dumps ... they didn't help much :p

Might be useful to add log to file in this function ( qp_inc/cron/_cron.funcs.php ) and seeing what it records :
Code: Select all
/**
 * Log a message from cron.
 * @param string Message
 * @param integer Level of importance. The higher the more important.
 *        (if $quiet (number of "-q" params passed to cron_exec.php)
 *         is higher than this, the message gets skipped)
 */
function cron_log( $message, $level = 0 )
{
    global $is_web, $quiet;

    if( $quiet > $level )
    {
        return;
    }

    if( $is_web )
    {
        echo '<p>'.$message.'</p>';
    }
    else
    
{
        echo "\n".$message."\n";
    }
}
 


Code: Select all
// Get next task to run in queue which has not started execution yet:
$sql = 'SELECT *
               FROM T_cron__task LEFT JOIN T_cron__log ON ctsk_ID = clog_ctsk_ID
             WHERE clog_ctsk_ID IS NULL
                AND ctsk_start_datetime <= '.$DB->quote( date2mysql($localtimenow) ).'
             ORDER BY ctsk_start_datetime ASC, ctsk_ID ASC
             LIMIT 1';


Amazes me that the sql above actually works, it *really* looks like it should fail .. every time :-S

¥
I may have opened the door but you entered of your own free will

Image Techno Babble II
Image Tacky Pad 3
Yabs
Dracone
User avatar
 
Posts: 896
Joined: Sat Nov 21, 2009 9:59 am

Re: Scheduled hitlog purging (cron)

Postby BushLeagueCritic » Sun Nov 21, 2010 4:53 pm

Ah, I think I'm actually beginning to see the problem. It has something to do with the added feature (at my request) to purge the cron__log table along with the hitlog table.

Once I get beyond the setting (7 days in my case) the join will create a dataset which includes entries from the task table that don't have a corresponding entry in the log table. That would explain why the very first task is being re-executed on the 8th day -- it's corresponding log entry was purged.

Me and my great log-purging ideas. :|

I see three solutions.
1) Remove the cron log purging routine from the Db::Prune function.
-Meh.
2) Prune the cron task table with the same frequency as the other pruning.
-Probably should be considered regardless.
3) Rewrite the sql that decides which tasks need executing (perhaps ignoring those that don't have a status of 'pending')
-Makes the most sense to me!
BushLeagueCritic
User avatar
 
Posts: 47
Joined: Wed Jul 21, 2010 2:50 pm

Re: Scheduled hitlog purging (cron)

Postby Yabs » Sun Nov 21, 2010 4:59 pm

4) move cron_pruning to hit's own job and fix the sql!

I look forward to your fully coded solution to the headache you shared with me :D

¥
I may have opened the door but you entered of your own free will

Image Techno Babble II
Image Tacky Pad 3
Yabs
Dracone
User avatar
 
Posts: 896
Joined: Sat Nov 21, 2009 9:59 am

Re: Scheduled hitlog purging (cron)

Postby BushLeagueCritic » Sun Nov 21, 2010 5:04 pm

I look forward to your fully coded solution to the headache you shared with me

I'm on it... like a duck on a june-bug. 8-)
BushLeagueCritic
User avatar
 
Posts: 47
Joined: Wed Jul 21, 2010 2:50 pm

Re: Scheduled hitlog purging (cron)

Postby BushLeagueCritic » Sun Nov 21, 2010 5:21 pm

Apparently, the task status (only viewable in the Back Office) actually relies on the cron__log table's lack of an entry to decide which jobs are 'pending.' That's a bit of a mess. :|
BushLeagueCritic
User avatar
 
Posts: 47
Joined: Wed Jul 21, 2010 2:50 pm

Re: Scheduled hitlog purging (cron)

Postby Yabs » Sun Nov 21, 2010 5:38 pm

Yabs wrote:I look forward to your fully coded solution to the headache you shared with me :D

¥


My status remains unchanged :D

¥
I may have opened the door but you entered of your own free will

Image Techno Babble II
Image Tacky Pad 3
Yabs
Dracone
User avatar
 
Posts: 896
Joined: Sat Nov 21, 2009 9:59 am


Return to You can sense the bug

Who is online

Users browsing this forum: No registered users and 1 guest

cron