Av an issue with some script or may be logic on howto I would want to capture the verbose of tcp dump to sniff traffic on a given port say ssh so i write some class # module sniff sniff class PrimarySpoof: * """ Primary spoof class. This class shall be used to read tcpdump output from the system it shall then pass this packet to a class variable called capturedPacket to make it available to other methods for further manupulation """* capturedPacket=None *def* __init__(self,port): """ initialize the port. """ self.port = port *def* initializeTcpdump(self): # you must have imported the os module """ This method will initialize tcpdump for the port indicated in the init. """ import os cmd = "tcpdump -nnvvXSs 1514 -i eth0 dst port %d"%self.port # test if it returns some output #cmd = "tcpdump -nnvvXSs 1514 -i eth0" PrimarySpoof.capturedPacket = os.system(cmd) *def* displayCapturedPacket(self): """ Display the raw packet. """ print PrimarySpoof.capturedPacket ################### end of part sample My question and problem is if i run several tcp dumps at the same time, will the output be in sync? Say (1) i first run a tcpdump to just get the headers i.e source and destination without the payload (2) run another tcpdump just to get the payload (3) Then I re-assemble the packet and >>> my fun things OR Get the entire verbose then get into the regex hell, remember this is a continual stream, picking out the headers pay load and the tail as well as doing the processing : well someone advice Kind Rgds *___________________________fun____________________________________ its ok child, He's heard you *
Ever heard of pastebin.com? put the code there it makes it very legible.. On Fri, Nov 12, 2010 at 3:10 PM, joe mwirigi <joemwirigi@gmail.com> wrote:
Av an issue with some script or may be logic on howto I would want to capture the verbose of tcp dump to sniff traffic on a given port say ssh so i write some class # module sniff sniff class PrimarySpoof: * """ Primary spoof class.
This class shall be used to read tcpdump output from the system it shall then pass this packet to a class variable called capturedPacket to make it available to other methods for further manupulation """* capturedPacket=None
*def* __init__(self,port): """ initialize the port.
""" self.port = port
*def* initializeTcpdump(self): # you must have imported the os module """ This method will initialize tcpdump for the port indicated in the init.
""" import os cmd = "tcpdump -nnvvXSs 1514 -i eth0 dst port %d"%self.port # test if it returns some output #cmd = "tcpdump -nnvvXSs 1514 -i eth0" PrimarySpoof.capturedPacket = os.system(cmd)
*def* displayCapturedPacket(self): """ Display the raw packet.
""" print PrimarySpoof.capturedPacket
################### end of part sample
My question and problem is if i run several tcp dumps at the same time, will the output be in sync?
Say (1) i first run a tcpdump to just get the headers i.e source and destination without the payload
(2) run another tcpdump just to get the payload
(3) Then I re-assemble the packet and >>> my fun things
OR Get the entire verbose then get into the regex hell, remember this is a continual stream, picking out the headers pay load and the tail as well as doing the processing :
well someone advice
Kind Rgds
*___________________________fun____________________________________ its ok child, He's heard you *
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
-- John Wesonga
@Joe, I'm rustic on the network side but will try to respond. Unless the packet is re-transmitted, you can only capture it once. Therefore in your application, grab the entire payload and load the stream into memory for parsing to an xml file. The nodes in your xml file will contain the layout of the packet as you know it eg headers. Using filters, select what you need to output. Can this work? HTHs.
Why not try something like? http://oss.coresecurity.com/projects/pcapy.html Otherwise you should probably implement threading to keep track of the different tcpdumps and their results. http://docs.python.org/library/threading.html and you are much more likely to get better answers at http://stackoverflow.com/ :) On 11/12/2010 03:10 PM, joe mwirigi wrote:
Av an issue with some script or may be logic on howto I would want to capture the verbose of tcp dump to sniff traffic on a given port say ssh so i write some class # module sniff sniff class PrimarySpoof: / """ Primary spoof class.
This class shall be used to read tcpdump output from the system it shall then pass this packet to a class variable called capturedPacket to make it available to other methods for further manupulation """/ capturedPacket=None
*def* __init__(self,port): """ initialize the port.
""" self.port = port
*def* initializeTcpdump(self): # you must have imported the os module """ This method will initialize tcpdump for the port indicated in the init.
""" import os cmd = "tcpdump -nnvvXSs 1514 -i eth0 dst port %d"%self.port # test if it returns some output #cmd = "tcpdump -nnvvXSs 1514 -i eth0" PrimarySpoof.capturedPacket = os.system(cmd)
*def* displayCapturedPacket(self): """ Display the raw packet.
""" print PrimarySpoof.capturedPacket
################### end of part sample
My question and problem is if i run several tcp dumps at the same time, will the output be in sync?
Say (1) i first run a tcpdump to just get the headers i.e source and destination without the payload
(2) run another tcpdump just to get the payload
(3) Then I re-assemble the packet and >>> my fun things
OR Get the entire verbose then get into the regex hell, remember this is a continual stream, picking out the headers pay load and the tail as well as doing the processing :
well someone advice
Kind Rgds
Joe - first dump the stream to a file and then process it. On 12 November 2010 13:37, Jonas | Lamu Software <jonas@lamusoftware.com>wrote:
Why not try something like? http://oss.coresecurity.com/projects/pcapy.html
Otherwise you should probably implement threading to keep track of the different tcpdumps and their results. http://docs.python.org/library/threading.html
and you are much more likely to get better answers at http://stackoverflow.com/ :)
On 11/12/2010 03:10 PM, joe mwirigi wrote:
Av an issue with some script or may be logic on howto I would want to capture the verbose of tcp dump to sniff traffic on a given port say ssh so i write some class # module sniff sniff class PrimarySpoof: / """ Primary spoof class.
This class shall be used to read tcpdump output from the system it shall then pass this packet to a class variable called capturedPacket to make it available to other methods for further manupulation """/ capturedPacket=None
*def* __init__(self,port): """ initialize the port.
""" self.port = port
*def* initializeTcpdump(self): # you must have imported the os module """ This method will initialize tcpdump for the port indicated in the init.
""" import os cmd = "tcpdump -nnvvXSs 1514 -i eth0 dst port %d"%self.port # test if it returns some output #cmd = "tcpdump -nnvvXSs 1514 -i eth0" PrimarySpoof.capturedPacket = os.system(cmd)
*def* displayCapturedPacket(self): """ Display the raw packet.
""" print PrimarySpoof.capturedPacket
################### end of part sample
My question and problem is if i run several tcp dumps at the same time, will the output be in sync?
Say (1) i first run a tcpdump to just get the headers i.e source and destination without the payload
(2) run another tcpdump just to get the payload
(3) Then I re-assemble the packet and >>> my fun things
OR Get the entire verbose then get into the regex hell, remember this is a continual stream, picking out the headers pay load and the tail as well as doing the processing :
well someone advice
Kind Rgds
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
@ jonas, pcapy worked well with centos, and py2.5 here am using scapy which is the same family with pcapy and impacket. Still scappy is still givingme issues with the payload though its super on all other packet manupulation stuff. With pcapy, i still have to go thro all the classes will get back if it favors me. @aki am trying the xml thingy but holding traffic to memmory .... am not sure its the way i must go thanks all trying out all suggestions *_______________________________________________________________ its ok child, He's heard you * On Fri, Nov 12, 2010 at 7:37 AM, Jonas | Lamu Software < jonas@lamusoftware.com> wrote:
Why not try something like? http://oss.coresecurity.com/projects/pcapy.html
Otherwise you should probably implement threading to keep track of the different tcpdumps and their results. http://docs.python.org/library/threading.html
and you are much more likely to get better answers at http://stackoverflow.com/ :)
On 11/12/2010 03:10 PM, joe mwirigi wrote:
Av an issue with some script or may be logic on howto I would want to capture the verbose of tcp dump to sniff traffic on a given port say ssh so i write some class # module sniff sniff class PrimarySpoof: / """ Primary spoof class.
This class shall be used to read tcpdump output from the system it shall then pass this packet to a class variable called capturedPacket to make it available to other methods for further manupulation """/ capturedPacket=None
*def* __init__(self,port): """ initialize the port.
""" self.port = port
*def* initializeTcpdump(self): # you must have imported the os module """ This method will initialize tcpdump for the port indicated in the init.
""" import os cmd = "tcpdump -nnvvXSs 1514 -i eth0 dst port %d"%self.port # test if it returns some output #cmd = "tcpdump -nnvvXSs 1514 -i eth0" PrimarySpoof.capturedPacket = os.system(cmd)
*def* displayCapturedPacket(self): """ Display the raw packet.
""" print PrimarySpoof.capturedPacket
################### end of part sample
My question and problem is if i run several tcp dumps at the same time, will the output be in sync?
Say (1) i first run a tcpdump to just get the headers i.e source and destination without the payload
(2) run another tcpdump just to get the payload
(3) Then I re-assemble the packet and >>> my fun things
OR Get the entire verbose then get into the regex hell, remember this is a continual stream, picking out the headers pay load and the tail as well as doing the processing :
well someone advice
Kind Rgds
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
@Joe, just a question for all you seasoned guys on programming. The maximum payload for the dump file would be say eg 10 Mbytes per second? Processing in memory is faster than file. What could be the reason not to use memory? Rgds. On Fri, Nov 12, 2010 at 3:46 PM, joe mwirigi <joemwirigi@gmail.com> wrote:
@aki am trying the xml thingy but holding traffic to memmory .... am not sure its the way i must go
thanks all trying out all suggestions
@aki, sorry i had got it all wrong, the saving to file was and is what i dont intend to todo *_______________________________________________________________ its ok child, He's heard you * On Fri, Nov 12, 2010 at 7:51 AM, aki <aki275@gmail.com> wrote:
@Joe, just a question for all you seasoned guys on programming. The maximum payload for the dump file would be say eg 10 Mbytes per second? Processing in memory is faster than file. What could be the reason not to use memory?
Rgds.
On Fri, Nov 12, 2010 at 3:46 PM, joe mwirigi <joemwirigi@gmail.com> wrote:
@aki am trying the xml thingy but holding traffic to memmory .... am not sure its the way i must go
thanks all trying out all suggestions
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
@andrew its lots of data am dealing with and all filtering has to be done on the fly *_______________________________________________________________ its ok child, He's heard you * On Fri, Nov 12, 2010 at 7:54 AM, joe mwirigi <joemwirigi@gmail.com> wrote:
@aki, sorry i had got it all wrong, the saving to file was and is what i dont intend to todo
*_______________________________________________________________ its ok child, He's heard you *
On Fri, Nov 12, 2010 at 7:51 AM, aki <aki275@gmail.com> wrote:
@Joe, just a question for all you seasoned guys on programming. The maximum payload for the dump file would be say eg 10 Mbytes per second? Processing in memory is faster than file. What could be the reason not to use memory?
Rgds.
On Fri, Nov 12, 2010 at 3:46 PM, joe mwirigi <joemwirigi@gmail.com>wrote:
@aki am trying the xml thingy but holding traffic to memmory .... am not sure its the way i must go
thanks all trying out all suggestions
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
@Joe, sorry me too, I was bit confused there. Btw, you have chosen a huge challenge with decoding the raw data. I just went through some old docs, The Byte Offsets/Bit Offsets in the Ip Hreader are crazy. And you have to do the same for UDP. I see where @Jonas come up with the word thread. Seems socket programming is processor intensive. This is quite interesting, hope you don't mind if I follow as a spectator. :-) On Fri, Nov 12, 2010 at 3:54 PM, joe mwirigi <joemwirigi@gmail.com> wrote:
@aki, sorry i had got it all wrong, the saving to file was and is what i dont intend to todo
*_______________________________________________________________ its ok child, He's heard you *
@aki, am not intending to go to the hard bits, all i want is to process the payload for "scapy" which is an elaborate module on packet processing and then have scapy deal with everything else on layer 2 and layer 3. Both tcpdump, scapy and the cousins pcapy and impacket all utilize the pcap library which is a "C" library and is well tuned. in short am just using other peolple's work to pull this one. *_______________________________________________________________ its ok child, He's heard you * On Fri, Nov 12, 2010 at 8:07 AM, aki <aki275@gmail.com> wrote:
@Joe, sorry me too, I was bit confused there. Btw, you have chosen a huge challenge with decoding the raw data. I just went through some old docs, The Byte Offsets/Bit Offsets in the Ip Hreader are crazy. And you have to do the same for UDP. I see where @Jonas come up with the word thread. Seems socket programming is processor intensive. This is quite interesting, hope you don't mind if I follow as a spectator. :-)
On Fri, Nov 12, 2010 at 3:54 PM, joe mwirigi <joemwirigi@gmail.com> wrote:
@aki, sorry i had got it all wrong, the saving to file was and is what i dont intend to todo
*_______________________________________________________________ its ok child, He's heard you *
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
@Joe, am I glad to hear that, phew! That decoding bit looked too much. Thanks for sharing your initial question and hope you get a solution. Asante. :-) On Fri, Nov 12, 2010 at 4:25 PM, joe mwirigi <joemwirigi@gmail.com> wrote:
@aki, am not intending to go to the hard bits, all i want is to process the payload for "scapy" which is an elaborate module on packet processing and then have scapy deal with everything else on layer 2 and layer 3. Both tcpdump, scapy and the cousins pcapy and impacket all utilize the pcap library which is a "C" library and is well tuned.
in short am just using other peolple's work to pull this one.
@aki aren't you in favour of doing everything from scratch ;) re-inventing the wheel et al On 11/12/2010 04:36 PM, aki wrote:
@Joe, am I glad to hear that, phew! That decoding bit looked too much. Thanks for sharing your initial question and hope you get a solution. Asante. :-) On Fri, Nov 12, 2010 at 4:25 PM, joe mwirigi <joemwirigi@gmail.com <mailto:joemwirigi@gmail.com>> wrote:
@aki, am not intending to go to the hard bits, all i want is to process the payload for "scapy" which is an elaborate module on packet processing and then have scapy deal with everything else on layer 2 and layer 3. Both tcpdump, scapy and the cousins pcapy and impacket all utilize the pcap library which is a "C" library and is well tuned.
in short am just using other peolple's work to pull this one.
Hello @Jonas. I now know you've gotten me competely wrong on the re-invention thing but am glad you brought it up. To clarify to developers, when I wrote about the said subject what I meant was about concepts. I know everyone uses frameworks or platforms to make their work load easier which is grand. We should be using all the tools available to get a fast turn around time on development. If you read my SMS gateway proposal, I suggested that devs should take Kannel and make it better for KE and Africa market thereby possibly creating an investment/business enviroments for themselves as the SMS market is set to grow further. When I asked you about the automation of the kenyan domain registration system , you wrote back that it would take much time and offered other alternatives. I understand time is not on our side but we are in 2010 and still do not have such a system in place. There are voids in our country on systems that developers have not ventured into or have entirely avoided. So while we can place an imported code system in place which I have no problem with, are we saying that we cannot create locally to try to compete with such systems? I say build it on open source if it is your area of development if you wish, but please build it because Kenya needs it. Compare the skills + labour intensive nature of the masai-market world. These Artisans are no different from developers. Yet when you go each week to such markets, you can find all the products you need. Such artisans do not know who is going to buy their products and at what competitive price but they still develop their products. Let us learn from them and create the next revolution of the software industry with kenyan apps/software for local or export consumption. Let us create the Kiondos of the software world and do something. Such things will not come from me but from you and others who are very good within the dev world. The rest of us have a lot of catching up to do therefore I can only urge you forward. Hope that clears the air. Nice weekend and catchup towards the end of next week. Rgds. On Fri, Nov 12, 2010 at 9:51 PM, Jonas | Lamu Software < jonas@lamusoftware.com> wrote:
@aki aren't you in favour of doing everything from scratch ;) re-inventing the wheel et al
Hi @aki, glad to get your view and clarification. Also happy that you are not trying to build your own Visual Studio ;) Regarding the DNS management platform. It is just an idea that will most likely never work because it would: 1. Require a large initial investment around 250,000USD 2. Probably never get approved as Kenic would have their say and unlikely that second country tld:s would get approved anyway 3. Mean that if Kenic would lower their yearly fee on .co.ke domains the idea (and 1) would be worth nothing 4. Require a fully automated Mobile Money solution that is not available Note that 4 is what is holding back a lot of the potential e-commerce in Kenya at the moment. Yes, there is A, B and C solutions but they are all based on the old Paypal version everyone was annoyed by where the end consumer is first required to register and top up their account on one website in order to make a purchase on the actual site. I have talked to A, B and C in frustration as it should not be a technical issue and turns out that it isn't - it is a legal one. Remember that Paypal's de facto success did not happen until this restriction was removed and ppl could pay directly without having to sign up for a Paypal account. But now this is getting very off subject so better create a new thread. On 11/12/2010 11:32 PM, aki wrote:
Hello @Jonas. I now know you've gotten me competely wrong on the re-invention thing but am glad you brought it up. To clarify to developers, when I wrote about the said subject what I meant was about concepts. I know everyone uses frameworks or platforms to make their work load easier which is grand. We should be using all the tools available to get a fast turn around time on development. If you read my SMS gateway proposal, I suggested that devs should take Kannel and make it better for KE and Africa market thereby possibly creating an investment/business enviroments for themselves as the SMS market is set to grow further. When I asked you about the automation of the kenyan domain registration system , you wrote back that it would take much time and offered other alternatives. I understand time is not on our side but we are in 2010 and still do not have such a system in place. There are voids in our country on systems that developers have not ventured into or have entirely avoided. So while we can place an imported code system in place which I have no problem with, are we saying that we cannot create locally to try to compete with such systems? I say build it on open source if it is your area of development if you wish, but please build it because Kenya needs it. Compare the skills + labour intensive nature of the masai-market world. These Artisans are no different from developers. Yet when you go each week to such markets, you can find all the products you need. Such artisans do not know who is going to buy their products and at what competitive price but they still develop their products. Let us learn from them and create the next revolution of the software industry with kenyan apps/software for local or export consumption. Let us create the Kiondos of the software world and do something. Such things will not come from me but from you and others who are very good within the dev world. The rest of us have a lot of catching up to do therefore I can only urge you forward. Hope that clears the air. Nice weekend and catchup towards the end of next week. Rgds. On Fri, Nov 12, 2010 at 9:51 PM, Jonas | Lamu Software <jonas@lamusoftware.com <mailto:jonas@lamusoftware.com>> wrote:
@aki aren't you in favour of doing everything from scratch ;) re-inventing the wheel et al
I though it was pastebin.ca and not pastebin.com...
Mwirigi - to process Mbps/Gbps payload data as fast as you're telling us you want, you may need a cluster or some other parallel distributed processing system which i don't think you have at the moment coz your question wouldn't appear on the list.. (and it's thus clear you can't be working with NSIS so it may take you time to get this going..) note also that unless your kernel is intentionally broken in special ways to bias fairness, your process would be quickly terminated as a memory/processor hog. So take my suggestion, first dump to file then process later. On 12 November 2010 14:25, joe mwirigi <joemwirigi@gmail.com> wrote:
@aki, am not intending to go to the hard bits, all i want is to process the payload for "scapy" which is an elaborate module on packet processing and then have scapy deal with everything else on layer 2 and layer 3. Both tcpdump, scapy and the cousins pcapy and impacket all utilize the pcap library which is a "C" library and is well tuned.
in short am just using other peolple's work to pull this one.
*_______________________________________________________________ its ok child, He's heard you *
On Fri, Nov 12, 2010 at 8:07 AM, aki <aki275@gmail.com> wrote:
@Joe, sorry me too, I was bit confused there. Btw, you have chosen a huge challenge with decoding the raw data. I just went through some old docs, The Byte Offsets/Bit Offsets in the Ip Hreader are crazy. And you have to do the same for UDP. I see where @Jonas come up with the word thread. Seems socket programming is processor intensive. This is quite interesting, hope you don't mind if I follow as a spectator. :-)
On Fri, Nov 12, 2010 at 3:54 PM, joe mwirigi <joemwirigi@gmail.com>wrote:
@aki, sorry i had got it all wrong, the saving to file was and is what i dont intend to todo
*_______________________________________________________________ its ok child, He's heard you *
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
@Andrew, sorry for asking this thing and this thread a learning lesson for me. Because the data is continuous, If he was to use a file and as the file stream writes, is it possible to read the same stream for sampling while writing? Rgds :-) On Fri, Nov 12, 2010 at 5:12 PM, Andrew Wachira <washirah@gmail.com> wrote:
Mwirigi - to process Mbps/Gbps payload data as fast as you're telling us you want, you may need a cluster or some other parallel distributed processing system which i don't think you have at the moment coz your question wouldn't appear on the list.. (and it's thus clear you can't be working with NSIS so it may take you time to get this going..) note also that unless your kernel is intentionally broken in special ways to bias fairness, your process would be quickly terminated as a memory/processor hog.
So take my suggestion, first dump to file then process later.
@andrew, am also trying to listen into very specific ports and dst IPs, I think this narrows the payload by far, and thus a small server machine is able to hundle this very comfortably *_______________________________________________________________ its ok child, He's heard you * On Fri, Nov 12, 2010 at 9:31 AM, aki <aki275@gmail.com> wrote:
@Andrew, sorry for asking this thing and this thread a learning lesson for me. Because the data is continuous, If he was to use a file and as the file stream writes, is it possible to read the same stream for sampling while writing? Rgds :-)
On Fri, Nov 12, 2010 at 5:12 PM, Andrew Wachira <washirah@gmail.com>wrote:
Mwirigi - to process Mbps/Gbps payload data as fast as you're telling us you want, you may need a cluster or some other parallel distributed processing system which i don't think you have at the moment coz your question wouldn't appear on the list.. (and it's thus clear you can't be working with NSIS so it may take you time to get this going..) note also that unless your kernel is intentionally broken in special ways to bias fairness, your process would be quickly terminated as a memory/processor hog.
So take my suggestion, first dump to file then process later.
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
@aki - yes; it's possible to read as you write - just keep track of your descriptors! On 12 November 2010 15:46, joe mwirigi <joemwirigi@gmail.com> wrote:
@andrew, am also trying to listen into very specific ports and dst IPs, I think this narrows the payload by far, and thus a small server machine is able to hundle this very comfortably
*_______________________________________________________________ its ok child, He's heard you *
On Fri, Nov 12, 2010 at 9:31 AM, aki <aki275@gmail.com> wrote:
@Andrew, sorry for asking this thing and this thread a learning lesson for me. Because the data is continuous, If he was to use a file and as the file stream writes, is it possible to read the same stream for sampling while writing? Rgds :-)
On Fri, Nov 12, 2010 at 5:12 PM, Andrew Wachira <washirah@gmail.com>wrote:
Mwirigi - to process Mbps/Gbps payload data as fast as you're telling us you want, you may need a cluster or some other parallel distributed processing system which i don't think you have at the moment coz your question wouldn't appear on the list.. (and it's thus clear you can't be working with NSIS so it may take you time to get this going..) note also that unless your kernel is intentionally broken in special ways to bias fairness, your process would be quickly terminated as a memory/processor hog.
So take my suggestion, first dump to file then process later.
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
@andrew, I've decided to split the packet as it comes in, and hand over the data to another method to proccess, does this have a negative impact on the network and (2) with higher loads will the same be felt by the network. i.e assuming traffic passes through this machine as a proxy *_______________________________________________________________ its ok child, He's heard you * On Fri, Nov 12, 2010 at 9:51 AM, Andrew Wachira <washirah@gmail.com> wrote:
@aki - yes; it's possible to read as you write - just keep track of your descriptors!
On 12 November 2010 15:46, joe mwirigi <joemwirigi@gmail.com> wrote:
@andrew, am also trying to listen into very specific ports and dst IPs, I think this narrows the payload by far, and thus a small server machine is able to hundle this very comfortably
*_______________________________________________________________ its ok child, He's heard you *
On Fri, Nov 12, 2010 at 9:31 AM, aki <aki275@gmail.com> wrote:
@Andrew, sorry for asking this thing and this thread a learning lesson for me. Because the data is continuous, If he was to use a file and as the file stream writes, is it possible to read the same stream for sampling while writing? Rgds :-)
On Fri, Nov 12, 2010 at 5:12 PM, Andrew Wachira <washirah@gmail.com>wrote:
Mwirigi - to process Mbps/Gbps payload data as fast as you're telling us you want, you may need a cluster or some other parallel distributed processing system which i don't think you have at the moment coz your question wouldn't appear on the list.. (and it's thus clear you can't be working with NSIS so it may take you time to get this going..) note also that unless your kernel is intentionally broken in special ways to bias fairness, your process would be quickly terminated as a memory/processor hog.
So take my suggestion, first dump to file then process later.
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
-- http://www.chromeexperiments.com
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
My Findings *def* initializeTcpdump(self): # you must have imported the os module """ This method will initialize tcpdump for the port indicated in the init. """ import os cmd = "tcpdump -nnvvXSs 1514 -i eth0 dst port %d"%self.port # test if it returns some output #cmd = "tcpdump -nnvvXSs 1514 -i eth0" PrimarySpoof.capturedPacket = os.system(cmd) I thought by merely assigning the system command to the object variable, I would woo the packet to it . After several tries and some rechecks, I found out that the Ovariable capturedPacket was returning empty so there are no string manipulations i could do on it. Unless it was possible to load the system verbose to this variable, this may not be the way to go... Leaves me with andrews idea of saving the file, or the pcapy/scapy way, myPacket = IP(dst = "somewhere.now")/TCP(dport=443,flags="S") thePayload = myPacket.payload.payload #work the rest *_______________________________________________________________ its ok child, He's heard you * On Fri, Nov 12, 2010 at 10:06 AM, joe mwirigi <joemwirigi@gmail.com> wrote:
@andrew, I've decided to split the packet as it comes in, and hand over the data to another method to proccess, does this have a negative impact on the network and (2) with higher loads will the same be felt by the network. i.e assuming traffic passes through this machine as a proxy
*_______________________________________________________________ its ok child, He's heard you *
On Fri, Nov 12, 2010 at 9:51 AM, Andrew Wachira <washirah@gmail.com>wrote:
@aki - yes; it's possible to read as you write - just keep track of your descriptors!
On 12 November 2010 15:46, joe mwirigi <joemwirigi@gmail.com> wrote:
@andrew, am also trying to listen into very specific ports and dst IPs, I think this narrows the payload by far, and thus a small server machine is able to hundle this very comfortably
*_______________________________________________________________ its ok child, He's heard you *
On Fri, Nov 12, 2010 at 9:31 AM, aki <aki275@gmail.com> wrote:
@Andrew, sorry for asking this thing and this thread a learning lesson for me. Because the data is continuous, If he was to use a file and as the file stream writes, is it possible to read the same stream for sampling while writing? Rgds :-)
On Fri, Nov 12, 2010 at 5:12 PM, Andrew Wachira <washirah@gmail.com>wrote:
Mwirigi - to process Mbps/Gbps payload data as fast as you're telling us you want, you may need a cluster or some other parallel distributed processing system which i don't think you have at the moment coz your question wouldn't appear on the list.. (and it's thus clear you can't be working with NSIS so it may take you time to get this going..) note also that unless your kernel is intentionally broken in special ways to bias fairness, your process would be quickly terminated as a memory/processor hog.
So take my suggestion, first dump to file then process later.
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
-- http://www.chromeexperiments.com
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
@andrew, being a newby in most of these things, networks and all, I may not put up any front. The lots of data to me may just be helos and handshakes of only a handfull of hardware, and apps communicating. All i want is to get some concept through. if it fails it will be proof enough that it wasnt a viable concept. :) I thot NSIS was the National Security Intelligent Service, when I do come across it or it falls in my path for unga I will learn it Much rgds *_______________________________________________________________ its ok child, He's heard you * On Fri, Nov 12, 2010 at 9:12 AM, Andrew Wachira <washirah@gmail.com> wrote:
Mwirigi - to process Mbps/Gbps payload data as fast as you're telling us you want, you may need a cluster or some other parallel distributed processing system which i don't think you have at the moment coz your question wouldn't appear on the list.. (and it's thus clear you can't be working with NSIS so it may take you time to get this going..) note also that unless your kernel is intentionally broken in special ways to bias fairness, your process would be quickly terminated as a memory/processor hog.
So take my suggestion, first dump to file then process later.
On 12 November 2010 14:25, joe mwirigi <joemwirigi@gmail.com> wrote:
@aki, am not intending to go to the hard bits, all i want is to process the payload for "scapy" which is an elaborate module on packet processing and then have scapy deal with everything else on layer 2 and layer 3. Both tcpdump, scapy and the cousins pcapy and impacket all utilize the pcap library which is a "C" library and is well tuned.
in short am just using other peolple's work to pull this one.
*_______________________________________________________________ its ok child, He's heard you *
On Fri, Nov 12, 2010 at 8:07 AM, aki <aki275@gmail.com> wrote:
@Joe, sorry me too, I was bit confused there. Btw, you have chosen a huge challenge with decoding the raw data. I just went through some old docs, The Byte Offsets/Bit Offsets in the Ip Hreader are crazy. And you have to do the same for UDP. I see where @Jonas come up with the word thread. Seems socket programming is processor intensive. This is quite interesting, hope you don't mind if I follow as a spectator. :-)
On Fri, Nov 12, 2010 at 3:54 PM, joe mwirigi <joemwirigi@gmail.com>wrote:
@aki, sorry i had got it all wrong, the saving to file was and is what i dont intend to todo
*_______________________________________________________________ its ok child, He's heard you *
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
-- http://www.chromeexperiments.com
_______________________________________________ Skunkworks mailing list Skunkworks@lists.my.co.ke http://lists.my.co.ke/cgi-bin/mailman/listinfo/skunkworks ------------ Skunkworks Rules http://my.co.ke/phpbb/viewtopic.php?f=24&t=94 ------------ Other services @ http://my.co.ke
participants (6)
-
aki -
Andrew Wachira -
Haggai Nyang -
joe mwirigi -
John Wesonga -
Jonas | Lamu Software