Are you looking for my non-technical blog?

This is now my technical-only blog, my non-technical blog is here.

14 September 2005

Manalaa Algorithm

Manalaa.net is an egyptian site that contains many useful services including a News Feed as weel as some Egyptian Blog Feed. They show posts from differen Egyptian Bloggers and show them and consolidate them in one portal. I this is done using a PHP script that can extract the Blog posts from the RSS or ATom feed and put it in some kind of DataBase - may be MySQL or PostgresSQL - and then show the DB contents on their page. But the script seems to use some special algorith in order to traverse the different blogs every certain period, as it sometimes show some blog post after others even if they are published earlier than them. So I think that the script may be visit some bloggers more frequent than others based on their activity or importance or whatever. Anyways I'd like to thank them for their effort and I with that they add some statistics for the more active bloggers, the most comented on posts, at what time of day do people post new articles more, etc.

3 comments:

  1. heheheheh special algorithm :-)

    la ya sidi no special algorithm wala 7aga, this is a stock drupal installation, my code contribution is writing an xslt based atom2rss converter since drupal 4.6 doesn't aggregate atom feeds.

    as for why you get these jumps, the aggregator is sorted according to post timestamps (converted to Cairo timezone), but many bloggers don't bother to properly configure their timezone so you get this weird ordering thing.

    another reason is actually that the server can't take it anymore, the aggregator recieves 3000 different visitors per day about a third of them are maniacs who check the aggregator every hour or so, this usually leads to the server failing, if it fails in the middle of checking for new posts then it will only aggregate some blogs and the rest will wait till the next hour.

    anyways I'm moving to a better server on friday, let's hope it works out.

    ReplyDelete
  2. as for statistics, this is a matter of crafting the proper sql queries, get drupal and help me out.

    ReplyDelete
  3. Hi
    I sure would like to give it a hand, it is really interresting
    The problem is that my FileSystem has been corrupted and my Linux partition has been diappeared. But I think I am gonna install Linux on my Home PC or even get a live Linux CD
    Any ways, just tell me how to help and I will for sure

    ReplyDelete