http://bulk.fefe.de/ An informative, and fairly technical, read on network and filesystem scalability bechmarks. The first 2-links are Unix-only (*BSD and Linux), the last link includes various Windows and Solaris versions. What comes out on top? The answer won't surprise you, neither will the identity of the worst-performing OS. What is shocking is how badly some otherwise decent OSes perform. The section on filesystems is also quite interesting: you thought non-journalled filesytems were faster than journalled ones? Think again. Quick FAQ on the article: O(n) means whatever is being measured proportionally increases as n increases, O(1) means performance is constant no matter how big n is. For instance: if you were measuring the amount of time it takes a webserver to serve requests, an O(n) performance means the time the webserver needs to serve a request increases proportionally to the number of requests; an O(1) performance means it takes the same amount of time for the webserver to serve 1 request as it would a gazillion bazillion requests. saidi
Big thanx for this one! --- On Sat, 9/26/09, saidimu apale <saidimu@gmail.com> wrote: From: saidimu apale <saidimu@gmail.com> Subject: [Skunkworks] scalability benchmarks To: "Skunkworks forum" <skunkworks@lists.my.co.ke> Date: Saturday, September 26, 2009, 7:40 AM http://bulk.fefe.de/ An informative, and fairly technical, read on network and filesystem scalability bechmarks. The first 2-links are Unix-only (*BSD and Linux), the last link includes various Windows and Solaris versions. What comes out on top? The answer won't surprise you, neither will the identity of the worst-performing OS. What is shocking is how badly some otherwise decent OSes perform. The section on filesystems is also quite interesting: you thought non-journalled filesytems were faster than journalled ones? Think again. Quick FAQ on the article: O(n) means whatever is being measured proportionally increases as n increases, O(1) means performance is constant no matter how big n is. For instance: if you were measuring the amount of time it takes a webserver to serve requests, an O(n) performance means the time the webserver needs to serve a request increases proportionally to the number of requests; an O(1) performance means it takes the same amount of time for the webserver to serve 1 request as it would a gazillion bazillion requests. saidi -----Inline Attachment Follows----- _______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
Wesley, what was your biggest surprise? Mine was the perfomance of journaled vs. non-journaled filesystems. It was a rather pleasant surprise. One of the biggest benefits, to me, of *nix is the variety of filesystems and the ability to mix and match the filesystems for maximum performance, e.g. one partition formatted with an fs that does well with many small files (but performs poorly for a few massive files) and another parition with an fs that does great with relatively few massive files (e.g. media files) but does badly with a huge number of small files (e.g. temp files from a webserver cache). saidi On Mon, Sep 28, 2009 at 11:48 AM, wesley kirinya <kiriinya2000@yahoo.com>wrote:
Big thanx for this one!
--- On *Sat, 9/26/09, saidimu apale <saidimu@gmail.com>* wrote:
From: saidimu apale <saidimu@gmail.com> Subject: [Skunkworks] scalability benchmarks To: "Skunkworks forum" <skunkworks@lists.my.co.ke> Date: Saturday, September 26, 2009, 7:40 AM
An informative, and fairly technical, read on network and filesystem scalability bechmarks. The first 2-links are Unix-only (*BSD and Linux), the last link includes various Windows and Solaris versions.
What comes out on top? The answer won't surprise you, neither will the identity of the worst-performing OS. What is shocking is how badly some otherwise decent OSes perform. The section on filesystems is also quite interesting: you thought non-journalled filesytems were faster than journalled ones? Think again.
Quick FAQ on the article: O(n) means whatever is being measured proportionally increases as n increases, O(1) means performance is constant no matter how big n is. For instance: if you were measuring the amount of time it takes a webserver to serve requests, an O(n) performance means the time the webserver needs to serve a request increases proportionally to the number of requests; an O(1) performance means it takes the same amount of time for the webserver to serve 1 request as it would a gazillion bazillion requests.
saidi
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke<http://us.mc1120.mail.yahoo.com/mc/compose?to=Skunkworks@lists.my.co.ke> http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
I wish I had some time to run these tests on a FreeBSD 7.0 and above. The freebsd appeared to oublaze the 2.6.21 kernel when I benchmarked some time last year - especially on the networking side. I suspected that the file-system was faster too - but someone oughtta check this out before swallowing the data.Also, the tests were done on a single CPU machine, there could be marked performance differences on smp machines. ________________________________ From: saidimu apale <saidimu@gmail.com> To: Skunkworks Forum <skunkworks@lists.my.co.ke> Sent: Monday, September 28, 2009 4:54:35 PM Subject: Re: [Skunkworks] scalability benchmarks Wesley, what was your biggest surprise? Mine was the perfomance of journaled vs. non-journaled filesystems. It was a rather pleasant surprise. One of the biggest benefits, to me, of *nix is the variety of filesystems and the ability to mix and match the filesystems for maximum performance, e.g. one partition formatted with an fs that does well with many small files (but performs poorly for a few massive files) and another parition with an fs that does great with relatively few massive files (e.g. media files) but does badly with a huge number of small files (e.g. temp files from a webserver cache). saidi On Mon, Sep 28, 2009 at 11:48 AM, wesley kirinya <kiriinya2000@yahoo.com> wrote:
Big thanx for this one!
--- On Sat, 9/26/09, saidimu apale <saidimu@gmail.com> wrote:
From: saidimu apale <saidimu@gmail.com> Subject: [Skunkworks] scalability benchmarks
To: "Skunkworks forum" <skunkworks@lists.my.co.ke> Date: Saturday, September 26, 2009, 7:40 AM
An informative, and fairly technical, read on network and filesystem scalability bechmarks. The first 2-links are Unix-only (*BSD and Linux), the last link includes various Windows and Solaris versions.
What comes out on top? The answer won't surprise you, neither will the identity of the worst-performing OS. What is shocking is how badly some otherwise decent OSes perform. The section on filesystems is also quite interesting: you thought non-journalled filesytems were faster than journalled ones? Think again.
Quick FAQ on the article: O(n) means whatever is being measured proportionally increases as n increases, O(1) means performance is constant no matter how big n is. For instance: if you were measuring the amount of time it takes a webserver to serve requests, an O(n) performance means the time the webserver needs to serve a request increases proportionally to the number of requests; an O(1) performance means it takes the same amount of time for the webserver to serve 1 request as it would a gazillion bazillion requests.
saidi
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks
Other services @ http://my.co.ke/ Other lists
Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce
Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
The lattest article is a presentation in 2006 and unfortunately it's not as detailed (since it's a presentation) as the 2003 article. The perfomance data measured in 2003 might vary as Bernard says given current date OS and hardware. Atleast the code seems available. I might run some test when I get a couple of free weeks. If I do, I'll post the results on the list (and start a flame war HAHAHAHA) 8~D --- On Tue, 9/29/09, Bernard Owuor <b_owuor@yahoo.com> wrote: From: Bernard Owuor <b_owuor@yahoo.com> Subject: Re: [Skunkworks] scalability benchmarks To: "Skunkworks Forum" <skunkworks@lists.my.co.ke> Date: Tuesday, September 29, 2009, 2:30 AM I wish I had some time to run these tests on a FreeBSD 7.0 and above. The freebsd appeared to oublaze the 2.6.21 kernel when I benchmarked some time last year - especially on the networking side. I suspected that the file-system was faster too - but someone oughtta check this out before swallowing the data. Also, the tests were done on a single CPU machine, there could be marked performance differences on smp machines. From: saidimu apale <saidimu@gmail.com> To: Skunkworks Forum <skunkworks@lists.my.co.ke> Sent: Monday, September 28, 2009 4:54:35 PM Subject: Re: [Skunkworks] scalability benchmarks Wesley, what was your biggest surprise? Mine was the perfomance of journaled vs. non-journaled filesystems. It was a rather pleasant surprise. One of the biggest benefits, to me, of *nix is the variety of filesystems and the ability to mix and match the filesystems for maximum performance, e.g. one partition formatted with an fs that does well with many small files (but performs poorly for a few massive files) and another parition with an fs that does great with relatively few massive files (e.g. media files) but does badly with a huge number of small files (e.g. temp files from a webserver cache). saidi On Mon, Sep 28, 2009 at 11:48 AM, wesley kirinya <kiriinya2000@yahoo.com> wrote: Big thanx for this one! --- On Sat, 9/26/09, saidimu apale <saidimu@gmail.com> wrote: From: saidimu apale <saidimu@gmail.com> Subject: [Skunkworks] scalability benchmarks To: "Skunkworks forum" <skunkworks@lists.my.co.ke> Date: Saturday, September 26, 2009, 7:40 AM http://bulk.fefe.de/ An informative, and fairly technical, read on network and filesystem scalability bechmarks. The first 2-links are Unix-only (*BSD and Linux), the last link includes various Windows and Solaris versions. What comes out on top? The answer won't surprise you, neither will the identity of the worst-performing OS. What is shocking is how badly some otherwise decent OSes perform. The section on filesystems is also quite interesting: you thought non-journalled filesytems were faster than journalled ones? Think again. Quick FAQ on the article: O(n) means whatever is being measured proportionally increases as n increases, O(1) means performance is constant no matter how big n is. For instance: if you were measuring the amount of time it takes a webserver to serve requests, an O(n) performance means the time the webserver needs to serve a request increases proportionally to the number of requests; an O(1) performance means it takes the same amount of time for the webserver to serve 1 request as it would a gazillion bazillion requests. saidi -----Inline Attachment Follows----- _______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke/ Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general -----Inline Attachment Follows----- _______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
There is a detailed writeup of the 2006 tests, I must have forgotten to post the link: http://bulk.fefe.de/lk2006/bench.html Regarding flame wars, it's cold over here so a little heat will do us much good ;) saidi ps Tech lists and flame wars are like "chanda na pete", it's what makes deary internet days entertaining. Plus, everyone on this list should already have purchased a flame-retardant suit. If not, I'm sure there's extra inventory lying somewhere on the 'net (bing/google it). On Tue, Sep 29, 2009 at 7:50 AM, wesley kirinya <kiriinya2000@yahoo.com>wrote:
The lattest article is a presentation in 2006 and unfortunately it's not as detailed (since it's a presentation) as the 2003 article. The perfomance data measured in 2003 might vary as Bernard says given current date OS and hardware. Atleast the code seems available. I might run some test when I get a couple of free weeks. If I do, I'll post the results on the list (and start a flame war HAHAHAHA)
8~D
--- On *Tue, 9/29/09, Bernard Owuor <b_owuor@yahoo.com>* wrote:
From: Bernard Owuor <b_owuor@yahoo.com> Subject: Re: [Skunkworks] scalability benchmarks To: "Skunkworks Forum" <skunkworks@lists.my.co.ke> Date: Tuesday, September 29, 2009, 2:30 AM
I wish I had some time to run these tests on a FreeBSD 7.0 and above. The freebsd appeared to oublaze the 2.6.21 kernel when I benchmarked some time last year - especially on the networking side. I suspected that the file-system was faster too - but someone oughtta check this out before swallowing the data. Also, the tests were done on a single CPU machine, there could be marked performance differences on smp machines.
------------------------------ *From:* saidimu apale <saidimu@gmail.com> *To:* Skunkworks Forum <skunkworks@lists.my.co.ke> *Sent:* Monday, September 28, 2009 4:54:35 PM *Subject:* Re: [Skunkworks] scalability benchmarks
Wesley, what was your biggest surprise? Mine was the perfomance of journaled vs. non-journaled filesystems. It was a rather pleasant surprise. One of the biggest benefits, to me, of *nix is the variety of filesystems and the ability to mix and match the filesystems for maximum performance, e.g. one partition formatted with an fs that does well with many small files (but performs poorly for a few massive files) and another parition with an fs that does great with relatively few massive files (e.g. media files) but does badly with a huge number of small files (e.g. temp files from a webserver cache).
saidi
On Mon, Sep 28, 2009 at 11:48 AM, wesley kirinya <kiriinya2000@yahoo.com<http://us.mc1120.mail.yahoo.com/mc/compose?to=kiriinya2000@yahoo.com>
wrote:
Big thanx for this one!
--- On *Sat, 9/26/09, saidimu apale <saidimu@gmail.com<http://us.mc1120.mail.yahoo.com/mc/compose?to=saidimu@gmail.com>
* wrote:
From: saidimu apale <saidimu@gmail.com<http://us.mc1120.mail.yahoo.com/mc/compose?to=saidimu@gmail.com>
Subject: [Skunkworks] scalability benchmarks To: "Skunkworks forum" <skunkworks@lists.my.co.ke<http://us.mc1120.mail.yahoo.com/mc/compose?to=skunkworks@lists.my.co.ke>
Date: Saturday, September 26, 2009, 7:40 AM
An informative, and fairly technical, read on network and filesystem scalability bechmarks. The first 2-links are Unix-only (*BSD and Linux), the last link includes various Windows and Solaris versions.
What comes out on top? The answer won't surprise you, neither will the identity of the worst-performing OS. What is shocking is how badly some otherwise decent OSes perform. The section on filesystems is also quite interesting: you thought non-journalled filesytems were faster than journalled ones? Think again.
Quick FAQ on the article: O(n) means whatever is being measured proportionally increases as n increases, O(1) means performance is constant no matter how big n is. For instance: if you were measuring the amount of time it takes a webserver to serve requests, an O(n) performance means the time the webserver needs to serve a request increases proportionally to the number of requests; an O(1) performance means it takes the same amount of time for the webserver to serve 1 request as it would a gazillion bazillion requests.
saidi
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke<http://us.mc1120.mail.yahoo.com/mc/compose?to=Skunkworks@lists.my.co.ke> http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke/ Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke<http://us.mc1120.mail.yahoo.com/mc/compose?to=Skunkworks@lists.my.co.ke> http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
Sorry, people. I did not intend to flame anyone - just stating what surprised me - and dented my faith in the soundness of those results. It's not that I don't appreciate the great job by Felix et al - and thanks Saidi for posting. I went through the data in a hurry - missing the fact that the round 2 freebsd benchmark was on 6.1 - the results were quite similar. FreeBSD 7.0 could be different, as windows 7 or Linux 2.6.28 & 2.6.30 - just check to make sure. ________________________________ From: saidimu apale <saidimu@gmail.com> To: Skunkworks Forum <skunkworks@lists.my.co.ke> Sent: Tuesday, September 29, 2009 1:59:15 PM Subject: Re: [Skunkworks] scalability benchmarks There is a detailed writeup of the 2006 tests, I must have forgotten to post the link: http://bulk.fefe.de/lk2006/bench.html Regarding flame wars, it's cold over here so a little heat will do us much good ;) saidi ps Tech lists and flame wars are like "chanda na pete", it's what makes deary internet days entertaining. Plus, everyone on this list should already have purchased a flame-retardant suit. If not, I'm sure there's extra inventory lying somewhere on the 'net (bing/google it). On Tue, Sep 29, 2009 at 7:50 AM, wesley kirinya <kiriinya2000@yahoo.com> wrote:
The lattest article is a presentation in 2006 and unfortunately it's not as detailed (since it's a presentation) as the 2003 article. The perfomance data measured in 2003 might vary as Bernard says given current date OS and hardware. Atleast the code seems available. I might run some test when I get a couple of free weeks. If I do, I'll post the results on the list (and start a flame war HAHAHAHA)
8~D
--- On Tue, 9/29/09, Bernard Owuor <b_owuor@yahoo.com> wrote:
From: Bernard Owuor <b_owuor@yahoo.com>
Subject: Re: [Skunkworks] scalability benchmarks
To: "Skunkworks Forum" <skunkworks@lists.my.co.ke> Date: Tuesday, September 29, 2009, 2:30 AM
I wish I had some time to run these tests on a FreeBSD 7.0 and above. The freebsd appeared to oublaze the 2.6.21 kernel when I benchmarked some time last year - especially on the networking side. I suspected that the file-system was faster too - but someone oughtta check this out before swallowing the data. Also, the tests were done on a single CPU machine, there could be marked performance differences on smp machines.
________________________________ From: saidimu apale <saidimu@gmail.com>
To: Skunkworks Forum <skunkworks@lists.my.co.ke> Sent: Monday, September 28, 2009 4:54:35 PM Subject: Re: [Skunkworks] scalability benchmarks
Wesley, what was your biggest surprise?
Mine was the perfomance of journaled vs. non-journaled filesystems. It was a rather pleasant surprise. One of the biggest benefits, to me, of *nix is the variety of filesystems and the ability to mix and match the filesystems for maximum performance, e.g. one partition formatted with an fs that does well with many small files (but performs poorly for a few massive files) and another parition with an fs that does great with relatively few massive files (e.g. media files) but does badly with a huge number of small files (e.g. temp files from a webserver cache).
saidi
On Mon, Sep 28, 2009 at 11:48 AM, wesley kirinya <kiriinya2000@yahoo.com> wrote:
Big thanx for this one!
--- On Sat, 9/26/09, saidimu apale <saidimu@gmail.com> wrote:
From: saidimu apale <saidimu@gmail.com>
>> Subject: [Skunkworks] scalability benchmarks To: "Skunkworks forum" <skunkworks@lists.my.co.ke> >> Date: Saturday, September 26, 2009, 7:40 AM
An informative, and fairly technical, read on network and filesystem scalability bechmarks. The first 2-links are Unix-only (*BSD and Linux), the last link includes various Windows and Solaris versions.
What comes out on top? The answer won't surprise you, neither will the identity of the worst-performing OS. What is shocking is how badly some otherwise decent OSes perform. The section on filesystems is also quite interesting: you thought non-journalled filesytems were faster than journalled ones? Think again.
Quick FAQ on the article: O(n) means whatever is being measured proportionally increases as n increases, O(1) means performance is constant no matter how big n is. For instance: if you were measuring the amount of time it takes a webserver to serve requests, an O(n) performance means the time the webserver needs to serve a request increases proportionally to the number of requests; an O(1) performance means it takes the same amount of time for the webserver to serve 1 request as it would a gazillion bazillion requests.
saidi
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke/
>> Other lists
Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks
Other services @ http://my.co.ke/ Other lists
Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce
Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
_______________________________________________
Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
LOL, you didn't flame anyone. Usually, even as Felix experienced, results from such tests start 'cyber wars', therefore if I (or any skunk(ette)) did some tests and posted them it might lead to a flame war. O~) --- On Wed, 9/30/09, Bernard Owuor <b_owuor@yahoo.com> wrote: From: Bernard Owuor <b_owuor@yahoo.com> Subject: Re: [Skunkworks] scalability benchmarks To: "Skunkworks Forum" <skunkworks@lists.my.co.ke> Date: Wednesday, September 30, 2009, 6:51 AM Sorry, people. I did not intend to flame anyone - just stating what surprised me - and dented my faith in the soundness of those results. It's not that I don't appreciate the great job by Felix et al - and thanks Saidi for posting. I went through the data in a hurry - missing the fact that the round 2 freebsd benchmark was on 6.1 - the results were quite similar. FreeBSD 7.0 could be different, as windows 7 or Linux 2.6.28 & 2.6.30 - just check to make sure. From: saidimu apale <saidimu@gmail.com> To: Skunkworks Forum <skunkworks@lists.my.co.ke> Sent: Tuesday, September 29, 2009 1:59:15 PM Subject: Re: [Skunkworks] scalability benchmarks There is a detailed writeup of the 2006 tests, I must have forgotten to post the link: http://bulk.fefe.de/lk2006/bench.html Regarding flame wars, it's cold over here so a little heat will do us much good ;) saidi ps Tech lists and flame wars are like "chanda na pete", it's what makes deary internet days entertaining. Plus, everyone on this list should already have purchased a flame-retardant suit. If not, I'm sure there's extra inventory lying somewhere on the 'net (bing/google it). On Tue, Sep 29, 2009 at 7:50 AM, wesley kirinya <kiriinya2000@yahoo.com> wrote: The lattest article is a presentation in 2006 and unfortunately it's not as detailed (since it's a presentation) as the 2003 article. The perfomance data measured in 2003 might vary as Bernard says given current date OS and hardware. Atleast the code seems available. I might run some test when I get a couple of free weeks. If I do, I'll post the results on the list (and start a flame war HAHAHAHA) 8~D --- On Tue, 9/29/09, Bernard Owuor <b_owuor@yahoo.com> wrote: From: Bernard Owuor <b_owuor@yahoo.com> Subject: Re: [Skunkworks] scalability benchmarks To: "Skunkworks Forum" <skunkworks@lists.my.co.ke> Date: Tuesday, September 29, 2009, 2:30 AM I wish I had some time to run these tests on a FreeBSD 7.0 and above. The freebsd appeared to oublaze the 2.6.21 kernel when I benchmarked some time last year - especially on the networking side. I suspected that the file-system was faster too - but someone oughtta check this out before swallowing the data. Also, the tests were done on a single CPU machine, there could be marked performance differences on smp machines. From: saidimu apale <saidimu@gmail.com> To: Skunkworks Forum <skunkworks@lists.my.co.ke> Sent: Monday, September 28, 2009 4:54:35 PM Subject: Re: [Skunkworks] scalability benchmarks Wesley, what was your biggest surprise? Mine was the perfomance of journaled vs. non-journaled filesystems. It was a rather pleasant surprise. One of the biggest benefits, to me, of *nix is the variety of filesystems and the ability to mix and match the filesystems for maximum performance, e.g. one partition formatted with an fs that does well with many small files (but performs poorly for a few massive files) and another parition with an fs that does great with relatively few massive files (e.g. media files) but does badly with a huge number of small files (e.g. temp files from a webserver cache). saidi On Mon, Sep 28, 2009 at 11:48 AM, wesley kirinya <kiriinya2000@yahoo.com> wrote: Big thanx for this one! --- On Sat, 9/26/09, saidimu apale <saidimu@gmail.com> wrote: From: saidimu apale <saidimu@gmail.com> Subject: [Skunkworks] scalability benchmarks To: "Skunkworks forum" <skunkworks@lists.my.co.ke> Date: Saturday, September 26, 2009, 7:40 AM http://bulk.fefe.de/ An informative, and fairly technical, read on network and filesystem scalability bechmarks. The first 2-links are Unix-only (*BSD and Linux), the last link includes various Windows and Solaris versions. What comes out on top? The answer won't surprise you, neither will the identity of the worst-performing OS. What is shocking is how badly some otherwise decent OSes perform. The section on filesystems is also quite interesting: you thought non-journalled filesytems were faster than journalled ones? Think again. Quick FAQ on the article: O(n) means whatever is being measured proportionally increases as n increases, O(1) means performance is constant no matter how big n is. For instance: if you were measuring the amount of time it takes a webserver to serve requests, an O(n) performance means the time the webserver needs to serve a request increases proportionally to the number of requests; an O(1) performance means it takes the same amount of time for the webserver to serve 1 request as it would a gazillion bazillion requests. saidi -----Inline Attachment Follows----- _______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke/ Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general -----Inline Attachment Follows----- _______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke/ Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general _______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general -----Inline Attachment Follows----- _______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
No worries Bernard, you didn't flame anyone at all. But it's nice to see that you're concerned. Having said that, flaming is a fact of internet-life (and quite amusing at that). The best parts of Slashdot are the comments, it's amazing how creatively flammable some people are. Positively sublime. Regarding the results, I normally try to stay away from OS-comparisons (though the exception to this is one OS that I avoid like swine-flu). I prefer looking at filesystems, which are much less kernel-dependent. Unfortunately network stacks are more closely tied to the OS but my experience has been such that I maxed out disk and filesystem systems *before* maxing out the network systems. saidi On Wed, Sep 30, 2009 at 1:57 AM, wesley kirinya <kiriinya2000@yahoo.com>wrote:
LOL, you didn't flame anyone. Usually, even as Felix experienced, results from such tests start 'cyber wars', therefore if I (or any skunk(ette)) did some tests and posted them it might lead to a flame war.
O~)
--- On *Wed, 9/30/09, Bernard Owuor <b_owuor@yahoo.com>* wrote:
From: Bernard Owuor <b_owuor@yahoo.com> Subject: Re: [Skunkworks] scalability benchmarks To: "Skunkworks Forum" <skunkworks@lists.my.co.ke> Date: Wednesday, September 30, 2009, 6:51 AM
Sorry, people. I did not intend to flame anyone - just stating what surprised me - and dented my faith in the soundness of those results. It's not that I don't appreciate the great job by Felix et al - and thanks Saidi for posting. I went through the data in a hurry - missing the fact that the round 2 freebsd benchmark was on 6.1 - the results were quite similar. FreeBSD 7.0 could be different, as windows 7 or Linux 2.6.28 & 2.6.30 - just check to make sure.
------------------------------ *From:* saidimu apale <saidimu@gmail.com> *To:* Skunkworks Forum <skunkworks@lists.my.co.ke> *Sent:* Tuesday, September 29, 2009 1:59:15 PM *Subject:* Re: [Skunkworks] scalability benchmarks
There is a detailed writeup of the 2006 tests, I must have forgotten to post the link: http://bulk.fefe.de/lk2006/bench.html Regarding flame wars, it's cold over here so a little heat will do us much good ;)
saidi
ps Tech lists and flame wars are like "chanda na pete", it's what makes deary internet days entertaining. Plus, everyone on this list should already have purchased a flame-retardant suit. If not, I'm sure there's extra inventory lying somewhere on the 'net (bing/google it).
On Tue, Sep 29, 2009 at 7:50 AM, wesley kirinya <kiriinya2000@yahoo.com<http://us.mc1120.mail.yahoo.com/mc/compose?to=kiriinya2000@yahoo.com>
wrote:
The lattest article is a presentation in 2006 and unfortunately it's not as detailed (since it's a presentation) as the 2003 article. The perfomance data measured in 2003 might vary as Bernard says given current date OS and hardware. Atleast the code seems available. I might run some test when I get a couple of free weeks. If I do, I'll post the results on the list (and start a flame war HAHAHAHA)
8~D
--- On *Tue, 9/29/09, Bernard Owuor <b_owuor@yahoo.com<http://us.mc1120.mail.yahoo.com/mc/compose?to=b_owuor@yahoo.com>
* wrote:
From: Bernard Owuor <b_owuor@yahoo.com<http://us.mc1120.mail.yahoo.com/mc/compose?to=b_owuor@yahoo.com>>
Subject: Re: [Skunkworks] scalability benchmarks To: "Skunkworks Forum" <skunkworks@lists.my.co.ke<http://us.mc1120.mail.yahoo.com/mc/compose?to=skunkworks@lists.my.co.ke>
Date: Tuesday, September 29, 2009, 2:30 AM
I wish I had some time to run these tests on a FreeBSD 7.0 and above. The freebsd appeared to oublaze the 2.6.21 kernel when I benchmarked some time last year - especially on the networking side. I suspected that the file-system was faster too - but someone oughtta check this out before swallowing the data. Also, the tests were done on a single CPU machine, there could be marked performance differences on smp machines.
------------------------------ *From:* saidimu apale <saidimu@gmail.com<http://us.mc1120.mail.yahoo.com/mc/compose?to=saidimu@gmail.com>
*To:* Skunkworks Forum <skunkworks@lists.my.co.ke<http://us.mc1120.mail.yahoo.com/mc/compose?to=skunkworks@lists.my.co.ke>
*Sent:* Monday, September 28, 2009 4:54:35 PM *Subject:* Re: [Skunkworks] scalability benchmarks
Wesley, what was your biggest surprise? Mine was the perfomance of journaled vs. non-journaled filesystems. It was a rather pleasant surprise. One of the biggest benefits, to me, of *nix is the variety of filesystems and the ability to mix and match the filesystems for maximum performance, e.g. one partition formatted with an fs that does well with many small files (but performs poorly for a few massive files) and another parition with an fs that does great with relatively few massive files (e.g. media files) but does badly with a huge number of small files (e.g. temp files from a webserver cache).
saidi
On Mon, Sep 28, 2009 at 11:48 AM, wesley kirinya <kiriinya2000@yahoo.com<http://us.mc1120.mail.yahoo.com/mc/compose?to=kiriinya2000@yahoo.com>
wrote:
Big thanx for this one!
--- On *Sat, 9/26/09, saidimu apale <saidimu@gmail.com<http://us.mc1120.mail.yahoo.com/mc/compose?to=saidimu@gmail.com>
* wrote:
From: saidimu apale <saidimu@gmail.com<http://us.mc1120.mail.yahoo.com/mc/compose?to=saidimu@gmail.com>
Subject: [Skunkworks] scalability benchmarks To: "Skunkworks forum" <skunkworks@lists.my.co.ke<http://us.mc1120.mail.yahoo.com/mc/compose?to=skunkworks@lists.my.co.ke>
Date: Saturday, September 26, 2009, 7:40 AM
An informative, and fairly technical, read on network and filesystem scalability bechmarks. The first 2-links are Unix-only (*BSD and Linux), the last link includes various Windows and Solaris versions.
What comes out on top? The answer won't surprise you, neither will the identity of the worst-performing OS. What is shocking is how badly some otherwise decent OSes perform. The section on filesystems is also quite interesting: you thought non-journalled filesytems were faster than journalled ones? Think again.
Quick FAQ on the article: O(n) means whatever is being measured proportionally increases as n increases, O(1) means performance is constant no matter how big n is. For instance: if you were measuring the amount of time it takes a webserver to serve requests, an O(n) performance means the time the webserver needs to serve a request increases proportionally to the number of requests; an O(1) performance means it takes the same amount of time for the webserver to serve 1 request as it would a gazillion bazillion requests.
saidi
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke<http://us.mc1120.mail.yahoo.com/mc/compose?to=Skunkworks@lists.my.co.ke> http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke/ Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke<http://us.mc1120.mail.yahoo.com/mc/compose?to=Skunkworks@lists.my.co.ke> http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke/ Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke<http://us.mc1120.mail.yahoo.com/mc/compose?to=Skunkworks@lists.my.co.ke> http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke<http://us.mc1120.mail.yahoo.com/mc/compose?to=Skunkworks@lists.my.co.ke> http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
Having said that, flaming is a fact of internet-life (and quite amusing at that). The best parts of Slashdot are the comments, it's amazing how creatively >flammable some people are. Positively sublime.
lol! These bench-marks target kernel hackers methinks - as most system integrators do not prioritize performance when selecting an OS. In a situation where CPU speed and RAM can overcomes differences in performance, most people go for: manageability (ease of use, required/available skill-set and support base, reliability etc), security and cost and availability. I never cared for the file-systems until recently - solid-state disks and eSATA etc are making their mark now. Whatever you did before, you were limited by the hardware seek - FS people had to settle for clever application-oriented caching - and that's what I think is still happening. Even now, I think it still is faster to store your data in the next machine - by gigabit ethernet - than to keep it in your own hard-drive! And, if you set up mysql-clustering or NFS, then you have effectively transfered all your worries from FS to the protocol stack. As for the protocol stack, have you every tried maxing out your gigabit ethernet? - just transfer (http, scp etc) 100MB of data and check how long it takes? Make sure you have tons of RAM - if it is too little, the disk will bottleneck, and repeat several times for good measure. Is it the 125MB/s you expected? ________________________________ From: saidimu apale <saidimu@gmail.com> To: Skunkworks Forum <skunkworks@lists.my.co.ke> Sent: Thursday, October 1, 2009 2:46:52 PM Subject: Re: [Skunkworks] scalability benchmarks No worries Bernard, you didn't flame anyone at all. But it's nice to see that you're concerned. Having said that, flaming is a fact of internet-life (and quite amusing at that). The best parts of Slashdot are the comments, it's amazing how creatively flammable some people are. Positively sublime. Regarding the results, I normally try to stay away from OS-comparisons (though the exception to this is one OS that I avoid like swine-flu). I prefer looking at filesystems, which are much less kernel-dependent. Unfortunately network stacks are more closely tied to the OS but my experience has been such that I maxed out disk and filesystem systems *before* maxing out the network systems. saidi On Wed, Sep 30, 2009 at 1:57 AM, wesley kirinya <kiriinya2000@yahoo.com> wrote:
LOL, you didn't flame anyone. Usually, even as Felix experienced, results from such tests start 'cyber wars', therefore if I (or any skunk(ette)) did some tests and posted them it might lead to a flame war.
O~)
--- On Wed, 9/30/09, Bernard Owuor <b_owuor@yahoo.com> wrote:
From: Bernard Owuor <b_owuor@yahoo.com>
Subject: Re: [Skunkworks] scalability benchmarks To: "Skunkworks Forum" <skunkworks@lists.my.co.ke> Date: Wednesday, September 30, 2009, 6:51 AM
Sorry, people. I did not intend to flame anyone - just stating what surprised me - and dented my faith in the soundness of those results. It's not that I don't appreciate the great job by Felix et al - and thanks Saidi for posting.
I went through the data in a hurry - missing the fact that the round 2 freebsd benchmark was on 6.1 - the results were quite similar. FreeBSD 7.0 could be different, as windows 7 or Linux 2.6.28 & 2.6.30 - just check to make sure.
________________________________ From: saidimu apale <saidimu@gmail.com>
To: Skunkworks Forum <skunkworks@lists.my.co.ke> Sent: Tuesday, September 29, 2009 1:59:15 PM Subject: Re: [Skunkworks] scalability benchmarks
There is a detailed writeup of the 2006 tests, I must have forgotten to post the link: http://bulk.fefe.de/lk2006/bench.html
Regarding flame wars, it's cold over here so a little heat will do us much good ;)
saidi
ps Tech lists and flame wars are like "chanda na pete", it's what makes deary internet days entertaining. Plus, everyone on this list should already have purchased a flame-retardant suit. If not, I'm sure there's extra inventory lying somewhere on the 'net (bing/google it).
On Tue, Sep 29, 2009 at 7:50 AM, wesley kirinya <kiriinya2000@yahoo.com> wrote:
The lattest article is a presentation in 2006 and unfortunately it's not as detailed (since it's a presentation) as the 2003 article. The perfomance data measured in 2003 might vary as Bernard says given current date OS and hardware. Atleast the code seems available. I might run some test when I get a couple of free weeks. If I do, I'll post the results on the list (and start a flame war HAHAHAHA)
8~D
--- On Tue, 9/29/09, Bernard Owuor <b_owuor@yahoo.com> wrote:
From: Bernard Owuor <b_owuor@yahoo.com>
Subject: Re: [Skunkworks] scalability benchmarks
To: "Skunkworks Forum" <skunkworks@lists.my.co.ke> Date: Tuesday, September 29, 2009, 2:30 AM
I wish I had some time to run these tests on a FreeBSD 7.0 and above. The freebsd appeared to oublaze the 2.6.21 kernel when I benchmarked some time last year - especially on the networking side. I suspected that the file-system was faster too - but someone oughtta check this out before swallowing the data. Also, the tests were done on a single CPU machine, there could be marked performance differences on smp machines.
From: saidimu apale <saidimu@gmail.com>
To: Skunkworks Forum <skunkworks@lists.my.co.ke> Sent: Monday, September 28, 2009 4:54:35 PM Subject: Re: [Skunkworks] scalability benchmarks
Wesley, what was your biggest surprise?
Mine was the perfomance of journaled vs. non-journaled filesystems. It was a rather pleasant surprise. One of the biggest benefits, to me, of *nix is the variety of filesystems and the ability to mix and match the filesystems for maximum performance, e.g. one partition formatted with an fs that does well with many small files (but performs poorly for a few massive files) and another parition with an fs that does great with relatively few massive files (e.g. media files) but does badly with a huge number of small files (e.g. temp files from a webserver cache).
saidi
On Mon, Sep 28, 2009 at 11:48 AM, wesley kirinya <kiriinya2000@yahoo.com> wrote:
Big thanx for this one!
--- On Sat, 9/26/09, saidimu apale <saidimu@gmail.com> wrote:
From: saidimu apale <saidimu@gmail.com> >>>>>> Subject: [Skunkworks] scalability benchmarks To: "Skunkworks forum" <skunkworks@lists.my.co.ke> >>>>>> Date: Saturday, September 26, 2009, 7:40 AM
An informative, and fairly technical, read on network and filesystem scalability bechmarks. The first 2-links are Unix-only (*BSD and Linux), the last link includes various Windows and Solaris versions.
What comes out on top? The answer won't surprise you, neither will the identity of the worst-performing OS. What is shocking is how badly some otherwise decent OSes perform. The section on filesystems is also quite interesting: you thought non-journalled filesytems were faster than journalled ones? Think again.
Quick FAQ on the article: O(n) means whatever is being measured proportionally increases as n increases, O(1) means performance is constant no matter how big n is. For instance: if you were measuring the amount of time it takes a webserver to serve requests, an O(n) performance means the time the webserver needs to serve a request increases proportionally to the number of requests; an O(1) performance means it takes the same amount of time for the webserver to serve 1 request as it would a gazillion bazillion requests.
saidi
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke/ >>>>>> Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke/
>> Other lists
Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce
>> Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke/
Other lists
Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce
Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
-----Inline Attachment Follows-----
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks
Other services @ http://my.co.ke Other lists
Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce
Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
_______________________________________________
Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks Other services @ http://my.co.ke Other lists ------------- Announce: http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks-announce Science: http://lists.my.co.ke/cgi-bin/mailman/listinfo/science kazi: http://lists.my.co.ke/cgi-bin/mailman/admin/kazi/general
participants (3)
-
Bernard Owuor -
saidimu apale -
wesley kirinya