Web Syndication......some help needed

While I'm ok with XML and reading/writing streams, I'm working on a test model that will aggregate some of the useful RSS links that am subscribed to. What am finding not easy to understand is how excatly the syndication process works, or rather the visual concept. Am looking for feedback/advice from someone who has built their own aggregator. I'm avoiding a practical test situation where too much polling would get me onto the restricted list of the servers. These are the questions, in no particular order: - Does a website publish an xml file as the syndication format and does the syndication file get stored on a local dir on the server awaiting request? - When the files are called with the http get command, how does the syndication format know of the last update and therefore releases as a stream? - Once the xml file has been downloaded onto the client machine, what distinguishes the latest from the older files and how does the client xml stream reader know about this update and which file to parse? Any help will be appreciated.

Not a specific solution but try downloading a copy of rssowl from http://www.rssowl.org/download .... power up you're favourite Java IDE, mines Eclipse(I'm also currently stuck trying to debug Ofbiz.... and between that and an urge to continue where I left from Call of duty: Black ops, my Sunday is totally eclipsed) and try and read between the XML setup. :-) On 1/9/11, aki <aki275@gmail.com> wrote:
While I'm ok with XML and reading/writing streams, I'm working on a test model that will aggregate some of the useful RSS links that am subscribed to. What am finding not easy to understand is how excatly the syndication process works, or rather the visual concept. Am looking for feedback/advice from someone who has built their own aggregator. I'm avoiding a practical test situation where too much polling would get me onto the restricted list of the servers. These are the questions, in no particular order:
- Does a website publish an xml file as the syndication format and does the syndication file get stored on a local dir on the server awaiting request?
- When the files are called with the http get command, how does the syndication format know of the last update and therefore releases as a stream?
- Once the xml file has been downloaded onto the client machine, what distinguishes the latest from the older files and how does the client xml stream reader know about this update and which file to parse?
Any help will be appreciated.

Patrick, there is an issue where RSSOwl encounters the following errors; <goog_176539164> - - An internal error occurred during: "". <goog_176539164> Uncaught Exception. db4o engine closed. <goog_176539164> - - An internal error occurred during: "". <goog_176539164> '/home/***/.rssowl2/.metadata/.plugins/org.rssowl.core/rssowl.db' is closed. close() was called or open() failed. <goog_176539164> <goog_176539164> This happens when a comp unexpectedly shuts down , eg during a blackout. Any info on how to solve this?

Probably a limitation with the embedded db4o database..... my guess is that you're running it on a .net platform, and if so, its probably on of the limitations of cross-platform usage On 1/9/11, Dennis Kioko <dmbuvi@gmail.com> wrote:
Patrick, there is an issue where RSSOwl encounters the following errors; <goog_176539164>
- - An internal error occurred during: "". <goog_176539164>
Uncaught Exception. db4o engine closed. <goog_176539164>
- - An internal error occurred during: "". <goog_176539164>
'/home/***/.rssowl2/.metadata/.plugins/org.rssowl.core/rssowl.db' is closed. close() was called or open() failed. <goog_176539164>
<goog_176539164> This happens when a comp unexpectedly shuts down , eg during a blackout. Any info on how to solve this?

My bad... you might want to try http://sourceforge.net/projects/rssowl/forums/forum/296909 On 1/9/11, Dennis Kioko <dmbuvi@gmail.com> wrote:
Running it on Linux

Thanks, restoring a back up db sorted me out

I thot you wanted ideas on sydicate dev On Sun, Jan 9, 2011 at 4:17 PM, Dennis Kioko <dmbuvi@gmail.com> wrote:
Thanks, restoring a back up db sorted me out _______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
-- www.golavish.com - The travel and leisure www.raccuddasys.com - code Development issues

Apologies for hijacking, Now, this is what Aki was saying "While I'm ok with XML and reading/writing streams, I'm working on a test model that will aggregate some of the useful RSS links that am subscribed to. What am finding not easy to understand is how excatly the syndication process works, or rather the visual concept. Am looking for feedback/advice from someone who has built their own aggregator. I'm avoiding a practical test situation where too much polling would get me onto the restricted list of the servers. These are the questions, in no particular order: - Does a website publish an xml file as the syndication format and does the syndication file get stored on a local dir on the server awaiting request? - When the files are called with the http get command, how does the syndication format know of the last update and therefore releases as a stream? - Once the xml file has been downloaded onto the client machine, what distinguishes the latest from the older files and how does the client xml stream reader know about this update and which file to parse? Any help will be appreciated."

Thanks @Patrick for the help though I'm no where near answering of the questions that I listed. The lack of responses is quite disappointing and have come to the realization that really I've been driving up the wrong alley with my programming queries, all these months with the hope that there are people who are good in this field. Web Syndication is not platform or language dependant. I can now take my question over the various programmer sites that am a member of and who actually know what they are doing. Very terrible, it is not easy to give up on a cause for KE but I see its a lost cause. Rgds.

Sounds pretty fateful, do you have a source control system? maybe if you were more detailed and provided the source code, detailing what you want to achieve and where you're hitting the wall, there might be more help, not just from KE but from all over the world. My 20 bits On 1/10/11, aki <aki275@gmail.com> wrote:
Thanks @Patrick for the help though I'm no where near answering of the questions that I listed.
The lack of responses is quite disappointing and have come to the realization that really I've been driving up the wrong alley with my programming queries, all these months with the hope that there are people who are good in this field. Web Syndication is not platform or language dependant. I can now take my question over the various programmer sites that am a member of and who actually know what they are doing. Very terrible, it is not easy to give up on a cause for KE but I see its a lost cause.
Rgds.

And for code, thought http://stackoverflow.com been better

@Patrick, I appreciated your response on reverse engineering ( using the IDE to look at the processes ) to understand the process but I can only reverse engineer something if I understand it well enough. :-) A general comment. After my disappointment with this thread, I sent a pm to a friend out there last night. He responded to me within 30 mins with a suggestion of a book that helped him on such issues. I bought the book online it has detailed everything that I need to know. Though it does not cover asp.net and c# ( my platform ), it is very throrough with the web syndication covering both RSS 2.0 and Atom 1.0 formats. It also covers a full subject on creating a very basic java based aggregator so I will interpret this for my platfrom. I'm finally off to a good start. I personally banned asking any more questions on this "wana-biashara customizers/tweekers <developers?> " list. Harsh? Web Syndication is as much of web programming as creating logon boxes and writing verification codes. Rgds. On Tue, Jan 11, 2011 at 12:58 AM, Patrick Kariuki <patrick.kariuki@gmail.com
wrote:
Sounds pretty fateful, do you have a source control system? maybe if you were more detailed and provided the source code, detailing what you want to achieve and where you're hitting the wall, there might be more help, not just from KE but from all over the world.
My 20 bits

@Aki, A couple of things 1) Si kwa ubaya, but you'll catch more bees with honey than with cowdung. Nobody is obliged to help you. Getting questions answered is not a right. Ergo if you insult people by calling them "wana-biashara customizers/tweekers <developers?> " i wonder what sort of responses you'll get when you need help 2) RSS and ATOM are very well documented protocols and a quick google could have sorted you out. 3) If you are on the .NET platform all the hard work has been done for you in the excellent System.ServiceModel.Syndication namespace (On .NET 4). It has classes to create and consume RSS 2 and Atom 1. You can find the documentation in MSDN or online here http://msdn.microsoft.com/en-us/library/system.servicemodel.syndication.aspx 4) The next time you write an email in anger, save it in the drafts and them come back after 24 hours and read it. You will find that being calmer will allow you to not send emails you might regret On Tue, Jan 11, 2011 at 8:47 AM, aki <aki275@gmail.com> wrote:
@Patrick, I appreciated your response on reverse engineering ( using the IDE to look at the processes ) to understand the process but I can only reverse engineer something if I understand it well enough. :-)
A general comment. After my disappointment with this thread, I sent a pm to a friend out there last night. He responded to me within 30 mins with a suggestion of a book that helped him on such issues. I bought the book online it has detailed everything that I need to know. Though it does not cover asp.net and c# ( my platform ), it is very throrough with the web syndication covering both RSS 2.0 and Atom 1.0 formats. It also covers a full subject on creating a very basic java based aggregator so I will interpret this for my platfrom. I'm finally off to a good start. I personally banned asking any more questions on this "wana-biashara customizers/tweekers <developers?> " list. Harsh? Web Syndication is as much of web programming as creating logon boxes and writing verification codes.
Rgds.
On Tue, Jan 11, 2011 at 12:58 AM, Patrick Kariuki < patrick.kariuki@gmail.com> wrote:
Sounds pretty fateful, do you have a source control system? maybe if you were more detailed and provided the source code, detailing what you want to achieve and where you're hitting the wall, there might be more help, not just from KE but from all over the world.
My 20 bits
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke

Hello @Rad, hata siyo kwa ubaya and the mail was not in anger, the term "wana-biashara customizers/tweekers <developers?> " is used not as an insult but excatly in the context that we should not expect any answers out of the business box. If any serious developer got offended, you have my apology in advance. There are many KE sites that have implemented RSS and one would hope that someone has at least taken the pains to understand the technology and therefore would be able to respond to a query. I'm subscribed to msdn, quite aware of the syndication classes, what google could provide in research, how others have implemented their code etc but will only use them as a reference in the future. My questions touched on the core of web syndication and whether anyone has done an aggregator, which would have been so helpful to engage on a regular basis and exchange ideas on things like creating web, system or platform alerts etc. I dare not ask a question on Soap or WDSL because I'm kind of done with the KE developer crowd. And to the "wana-biashara customizers/tweekers <developers?> " skunkers, you make a living off this so I have no right to make any comments however hope you can understand that we are on different levels of the developer subject. My possible entreprenuership on development will probably start in a few years time and along way I have learnt that it will be very very difficult to find core programmers, people who are knowledgeable and have a passion for what they do. Rgds. :-) On Tue, Jan 11, 2011 at 12:47 PM, Rad! <conradakunga@gmail.com> wrote:
@Aki,
A couple of things
1) Si kwa ubaya, but you'll catch more bees with honey than with cowdung. Nobody is obliged to help you. Getting questions answered is not a right. Ergo if you insult people by calling them "wana-biashara customizers/tweekers <developers?> " i wonder what sort of responses you'll get when you need help 2) RSS and ATOM are very well documented protocols and a quick google could have sorted you out. 3) If you are on the .NET platform all the hard work has been done for you in the excellent System.ServiceModel.Syndication namespace (On .NET 4). It has classes to create and consume RSS 2 and Atom 1. You can find the documentation in MSDN or online here http://msdn.microsoft.com/en-us/library/system.servicemodel.syndication.aspx 4) The next time you write an email in anger, save it in the drafts and them come back after 24 hours and read it. You will find that being calmer will allow you to not send emails you might regret
participants (5)
-
aki
-
Dennis Kioko
-
Frankline Chitwa
-
Patrick Kariuki
-
Rad!