Hping3

Network Tool Development with hping3
by Federico Biancuzzi
10/07/2004

In 1998, Salvatore Sanfilippo, also known as antirez, presented a new network scanner based on the IP ID field. While working as a security researcher, he started learning C to develop a tool called hping. That tool, then called hping2, is so good that it's still No. 6 in the Top 75 Security Tools of the Nmap Hackers mailing list.

However, Salvatore was not satisfied with the code, and he started a new project. The result is so good that it could influence the way network security specialists work. Until today, we have used tools such nmap or hping2 for specific tasks--automated scans with nmap, manual probes with hping2, and so on. People who needed particular features had only two options: writing a patch for an existing tool or developing a brand-new tool, probably based on libpcap, libnet, libdnet, or lib.

Maybe hping3 will revolutionize this approach. It's everything from a simple tool to a complete and scriptable framework for network analysis. The secret is the inclusion of a Tcl interpreter that interacts with the C core. Anyone, even a newbie programmer, can develop one of the famous nmap scan types with a simple Tcl script. The revolution has passed from writing a famous tool to developing a means of creating infinite features.

Federico Biancuzzi recently interviewed Salvatore by email about the development of hping3.

Could you present yourself?

Sure. Hello, I'm Salvatore Sanfilippo, an Italian open source developer in my spare time, mainly interested in security and programming languages. I'm 27 and work as a freelance consultant developing applications for my customers. Most applications are web-related, or low-level things like microcontrollers and network daemons.

How did your interest in network security start?

It all started when I installed Linux for the first time and joined IRC; this was around 1997 IIRC, I started with lame things like IRC wars :) Fortunately, after some time I got more interested in the technical side of the game, and started to study protocols and security. At some point I moved to Milan to work for a security company, where I found a lot of interesting people to talk with and to experiment with some new ideas. I didn't last more than four or five months in Milan because I'm from the south of Italy (Sicily) and found Milan not very good in terms of quality of life, but I learned many interesting things during this period.

If I'm not wrong, hping was your first open source project and one of your first programming projects. What have your learned developing the first version of hping?

When I wrote the first version of hping I had only five months of experience with the C language, so there was a lot to learn ... the first thing I discovered was the world of Unix system calls, and how a simple program like the first hping.c was only able to work on Linux and not on other Unix systems we had in the lab at Milan.

hping was also an experience regarding user interfaces. The first hping did less than many other programs already available, like the powerful ipsend, but the user interface was different, because hping is like ping in that everyone knows and can use it: you can see the replies from the target host, and to modify outgoing packets you just have to add or change command-line switches.

In my successive programming I have always tried to pay attention to user interfaces. I actually think that reinventing the wheel is not a bad idea if there is an effort to make programs simpler to use and more intuitive.

Why have you chosen the GNU General Public License to distribute your code?

Because I learned a lot using and looking at GPL code, and at the same time I don't like the idea that third parties can redistribute modified binary-only versions of hping.

What types of differences are there in the development process and code organization between hping2 and hping3?

The main difference is probably that hping2 was conceived as a tool, while hping3 adds to this tool a development system, something like a scriptable TCP/IP implementation in userspace. Another difference is that hping2 evolved incrementally, while in hping3 there was somewhat a goal from the beginning. At the moment, hping3 includes the support for hping2 command-line options; however, I'd like to remove that code from the C core and provide the same compatibility layer as an .htcl script.

About the code organization, hping3 is mainly a collection of libraries that may live apart from hping itself; hping2 instead is in the form of a single program that's very hard to reuse for something different.

From the user point of view, hping3 should be both simpler and more powerful, assuming that there will be two different classes of users. Programmers will be able to exploit the full power of a real programming language and a flexible packet construction/analysis sytem. On the other hand, it should be much easier for nondevelopers [to] run hping3 scripts developed by others than to use hping2. For example, one could develop a hping3 script to audit a firewall without doing all the common stuff by hand.

Why have you chosen Tcl?

Because I like "programmable programming languages" like Tcl and Lisp a lot. The programmer is free to reinvent the language, write new control structures, and so on. Tcl is very powerful, but for some reason few programmers fully understand it, so it's often regarded as a toy language. I use Tcl for everything not involving low-level things or speed. For the rest, I tend to use C.

What particular limit or feature does the language itself bring?

One of the best features that Tcl gives you is that you can specialize it so much to have a domain-specific language designed to play with packets. The limitation of all this flexibility is speed. For example, in theory you can use hping3 to write a firewalling engine, but in reality you can't go beyond a "prototype" because a firewall needs to be fast.

How does the TCL scripting engine communicate with the C core?

Tcl is designed to communicate with C, so it's pretty simple. The entire hping engine is exported to Tcl as a single Tcl command, so hping3 scripts are just Tcl scripts with a unique new command able to do a lot of new things related to packets and TCP/IP that are not possible with plain Tcl.

How does the C core work?

The C core is the sum of the hping2 code, used to provide command-line compatibility in this initial stage, and the new hping3 code. I'll focus on the latter. The core is comprised of different layers; the first layer is called ARS, which provides low-level packet construction and analysis. It works describing packets as different layers. For example, a packet can have two layers: IP and TCP. These layers are then "compiled"--the compilation will take care of computing checksums and dependencies between layers, so that, for example, the IP layer will get the total length field set according to the following layers, and so on. This way users of the library don't need to do too much and can just create a packet adding layers, and finally compiling the packet. The packet can be created as you like, but still the checksums, option paddings, and lengths will be OK. As I stated, ARS is also able to do the reverse--given a binary packet, it can be split into layers, modified, and then recompiled and possibly re-sent.

The higher level API is APD (ARS packet description). Basically this layer uses strings to represent ARS layers, so complex packets can be generated starting from human readable descriptions of layers like ip{saddr=1.2.3.4,daddr=5.6.7.8}+icmp{type=3,code=3} and so on, or binary packets can be translated into the string representation.

Given that APD packets are just strings, they are very natural to bind to Tcl, which is a language mainly based on strings.

Does hping3 take advantage of multi-CPU hardware?

No, for now at least, but there aren't plans to add support for threads because Tcl is mainly event driven in design. (Actually, Tcl is able to support threads pretty well; I just like more the event-driven way.)

I saw a lot of names in the credits page; could you tell us who did what?

Most of the contributions are limited to bug fixes or the addition of minor features, so I don't remember exactly who did what, with the exception of Nicolas Jombart, who provided significant help and code for the BSD port.

I tested hping3-alpha2 on FreeBSD and OpenBSD, and I found out that some scripts and docs are completely Linux oriented. Can we say that there's space in the development team for some BSD experts to improve the portability, especially on *BSD systems?

The goal of hping3 is to fully support at least Linux, *BSD, and Mac OS X. Currently most of the work is focused on Linux because that's the main development platform I use, but as happened for hping2, the BSD port will become more stable after some time. Actually hping3 is already working on BSD and Mac OS X, but there was far less testing on those platforms--what I mean is that most of the BSD port is already in the source code.

Do you plan to port hping3 to commercial systems like Solaris, Mac OS X, and Windows?

OS X for sure; it already works as well as the BSD port itself. I received some reports that hping3 is almost working on Solaris, which will eventually be semi-supported, but we will never claim to aim for Solaris support. Windows is a much more interesting target because of the big user base, and because there is already a working hping2 port (wiki.hping.org is where to go if you want to try it in your Win32 box), and most of hping3's OS-dependent code is either on the Tcl side, which is perfectly supported on Win32, or in functions for packet transmission and reception that can be easily extracted from hping2.

Are you looking for any developer? Which area would you like to add manpower to?

Now that hping3 is a programming environment, I'm looking for developers to provide interesting scripts written utilizing hping3 itself. Another interesting area is IPv6. hping3 is based on the ARS library, which is designed to be modular, so adding IPv6 shouldn't be hard.

Do you plan to create a repository for .htcl scripts created by the community like Nessus does?

Yes, it's one of my main goals once hping3 is widely used enough.

I've read some short docs on wiki.hping.org and some in the hping tarball. Are you looking for people to write more extensive documentation?

Yes! The goal of the Wiki is just that, to build a collaborative site in order to document hping directly online.

When do you think that hping3 will be stable enough to be released? I'm a little suspect that the release process could require a mythological time like Doom 3 did.

[Laughing] Actually it has already taken a lot of time, but now it's in the classic 98 percent done stage. The last 2 percent is very hard to do, but I think that the next time I don't have a big workload it will take another little jump. I hope that in 6 to 12 months at most I'll find the time to push hping to 3.0 stable.

Have you ever thought that hping3 could become an easier, alternative way to build low-level tools that today are written with C and libpcap, libnet, or libdnet?

This is the main goal of hping3 itself. Developers and security researchers are wasting a lot of time with low-level tools; they will go faster using advanced tools with a higher level of abstraction. If security researchers have better tools, they will be able to do more interesting research. Often programmers and researchers come up with a new idea, but if the efforts needed just to test whether the idea works in practice are too big, there is a good chance that the idea will remain untested for a long time.

In 1998 you posted on Bugtraq a short explanation of a new type of network scan called idle scan. It's amazing that it still works. Today we have networks and hosts that use multiple technologies to protect themselves, and then old and vulnerable hosts that permit spoofed scans. How can we make the Internet a better place if the gap among hosts' security keeps increasing? Trusting nothing? Paranoia by default?

In the case of the "idle scan," the problem is in the TCP/IP protocol: fortunately there is a fix you can apply in software (that is, unguessable IP IDs). Still, correct implementations of TCP/IP have this bug. So my guess is that Internet security should be more a collective and technological goal, rather than something focused on a few hosts on the Internet.

Part of the "secure host" idea is related to business, in my opinion. If you want to sell a security product, you have to create the (wrong) impression that your security fully depends on the amount of money you invest in security products. Unfortunately, the real world works in a different way--to be secure you need secure code for your applications, but these applications are in most cases developed by others, and code security in turn depends on software culture, libraries, programming languages, and other things related to the "community." Second, you need good and secure networking protocols, again something about the Internet community that you can't buy for yourself.

Last year nmap included the support for idle scan. What do you think about its implementation?

That's very good and smart, as nmap itself generally is. idle scan is simple to describe, but very hard to implement in a real-world program; it's not by accident that nmap is the first usable implementation of idle scan. I'm very happy that Fyodor [the author of nmap] did this great work, and I think he reached one of his goals: to make idle scan available enough to the masses in order to force TCP/IP implementations to fix this problem.

Have you ever noted that some of the most known open source networking tools (ettercap, ntop, hping, WinDump, and WinPcap) are developed by Italians? Why do you think we are so interested in networking and security?

It's hard to tell. I guess that in part it's because security is a stimulating thing. It's possible that another reason is that research is not well funded in Italy, yet there is still a lot of desire to do something new and innovative--and one of the rare fields where you can do a lot of interesting new things with little money is computer security.

I live in Italy like you, and I know that in our country it's nearly impossible to find any economic support for this type of project. How could the community help you? Are you looking for Internet services, hardware, or maybe a job?

I agree that in Italy it's very hard to get economic support :) But I think I found a way in order to have economic support (very minimal, but still enough to do a bit less work as a freelancer and invest more time in free software): advertising. I think that banners may provide enough economic support for many little projects, at least. hping gets quite a lot of unique visitors every day, so the best way to help hping is to put links to the hping web site where appropriate, if you own a web site.

Users will get software for free from your pages; I think that a little banner is acceptable. On the other hand, if open source software is funded by a company, it's possible the developer will not be free to implement what he likes, but what the company wants. Advertising doesn't have this problem; you get the money and can use the saved time to implement the features you want.

I also applied the advertising strategy to Visitors, a GPL weblog analyzer you can find at www.hping.org/visitors, and it is working pretty well even for this less known software.

Sendai's Story

Like many professional penetration testers, Sendai was not always the wholesome “ethical hacker” described in his employer's marketing material. In his youth, he stepped well over the line between questionable (grey hat) and flat-out illegal (black hat) behavior. Yet he never felt that he was doing anything wrong. Sendai did not intentionally damage systems, and was only trying to learn more about UNIX, networking, security, phone systems, and related technology. Yet the law might consider some of his actions to be unauthorized access, theft of services, wire fraud, copyright infringement, and trade secret theft. In the rare times that Sendai thought of this, he found solace in the words of the Mentor's Hacker Manifesto: “Yes, I am a criminal. My crime is that of curiosity.” Surely his innocent motives would prevent prosecution. Besides, his teenage arrogance assured him that the government and targeted corporations were too dumb to catch him.

This perception changed dramatically in 1989 and 1990 when the “Operation Sundevil” raids took place. Well-known security enthusiasts, including The Prophet, Knight Lightning, and Erik Bloodaxe, were raided and many more were indicted. The popular Phrack e-zine was shut down while its editor faced trial. Sendai worried that he, too, might be swept up in the persecution. After all, he had been active on some of the same bulletin boards as many suspects, performing similar activities. Sendai was never targeted, but those nine months of stress and paranoia changed his outlook on hacking. He was not exactly scared straight, but he ceased treating network intrusion as a game or casual hobby. In the following years, Sendai became much more disciplined about hiding his tracks through multiple layers of indirection, as well as always wiping logs, even when it was inconvenient. He also began to research his targets and methods much more extensively. Failing to fully understand a system could cause him to miss important defenses and lead to detection. A side effect of this more methodical approach to hacking is that Sendai substantially broadened his network security knowledge and skill set.

Sendai did not recognize the growing value of this skill set and clean record until he was offered the “ethical hacking” job at a well-known auditing firm. The burgeoning Internet was creating such intense demand for security professionals that the firm asked few questions about his past. Using his real name, they were unaware that he even used the hacker handle Sendai. He did have some reservations about commercializing his hobby, not wanting to be seen as a sell out. Despite these concerns, Sendai accepted the position immediately. It sure beat his previous technical support day job! Soon he was living in the security world during both days and nights. The job provided legitimate access to exciting enterprise technologies, and he could hone his hacking skills without risking arrest. Bragging about his exploits led to bonuses instead of jail time. Sendai had so much fun cracking into systems for money that he eventually ceased much of his nocturnal black hat network exploration.

Steve Jobs Speech To Graduates of Stanford University

Note: This is an excerpt of an incredibly inspiring speech by Steve Jobs (CEO of Apple and Pixar) delivered to graduates of Stanford University on June 12, 2005.

I am honored to be with you today at your commencement from one of the finest universities in the world. I never graduated from college. Truth be told, this is the closest I've ever gotten to a college graduation. Today I want to tell you three stories from my life. That's it. No big deal. Just three stories.
The first story is about connecting the dots.

I dropped out of Reed College after the first 6 months, but then stayed around as a drop-in for another 18 months or so before I really quit. So why did I drop out?

It started before I was born. My biological mother was a young, unwed college graduate student, and she decided to put me up for adoption. She felt very strongly that I should be adopted by college graduates, so everything was all set for me to be adopted at birth by a lawyer and his wife. Except that when I popped out they decided at the last minute that they really wanted a girl. So my parents, who were on a waiting list, got a call in the middle of the night asking: "We have an unexpected baby boy; do you want him?" They said: "Of course." My biological mother later found out that my mother had never graduated from college and that my father had never graduated from high school. She refused to sign the final adoption papers. She only relented a few months later when my parents promised that I would someday go to college.

And 17 years later I did go to college. But I naively chose a college that was almost as expensive as Stanford, and all of my working-class parents' savings were being spent on my college tuition. After six months, I couldn't see the value in it. I had no idea what I wanted to do with my life and no idea how college was going to help me figure it out. And here I was spending all of the money my parents had saved their entire life. So I decided to drop out and trust that it would all work out OK. It was pretty scary at the time, but looking back it was one of the best decisions I ever made. The minute I dropped out I could stop taking the required classes that didn't interest me, and begin dropping in on the ones that looked interesting.

It wasn't all romantic. I didn't have a dorm room, so I slept on the floor in friends' rooms, I returned coke bottles for the 5¢ deposits to buy food with, and I would walk the 7 miles across town every Sunday night to get one good meal a week at the Hare Krishna temple. I loved it. And much of what I stumbled into by following my curiosity and intuition turned out to be priceless later on. Let me give you one example:

Reed College at that time offered perhaps the best calligraphy instruction in the country. Throughout the campus every poster, every label on every drawer, was beautifully hand calligraphed. Because I had dropped out and didn't have to take the normal classes, I decided to take a calligraphy class to learn how to do this. I learned about serif and san serif typefaces, about varying the amount of space between different letter combinations, about what makes great typography great. It was beautiful, historical, artistically subtle in a way that science can't capture, and I found it fascinating.

None of this had even a hope of any practical application in my life. But ten years later, when we were designing the first Macintosh computer, it all came back to me. And we designed it all into the Mac. It was the first computer with beautiful typography. If I had never dropped in on that single course in college, the Mac would have never had multiple typefaces or proportionally spaced fonts. And since Windows just copied the Mac, its likely that no personal computer would have them. If I had never dropped out, I would have never dropped in on this calligraphy class, and personal computers might not have the wonderful typography that they do. Of course it was impossible to connect the dots looking forward when I was in college. But it was very, very clear looking backwards ten years later.

Again, you can't connect the dots looking forward; you can only connect them looking backwards. So you have to trust that the dots will somehow connect in your future. You have to trust in something - your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.

My second story is about love and loss.

I was lucky – I found what I loved to do early in life. Woz and I started Apple in my parents garage when I was 20. We worked hard, and in 10 years Apple had grown from just the two of us in a garage into a $2 billion company with over 4000 employees. We had just released our finest creation - the Macintosh - a year earlier, and I had just turned 30. And then I got fired. How can you get fired from a company you started? Well, as Apple grew we hired someone who I thought was very talented to run the company with me, and for the first year or so things went well. But then our visions of the future began to diverge and eventually we had a falling out. When we did, our Board of Directors sided with him. So at 30 I was out. And very publicly out. What had been the focus of my entire adult life was gone, and it was devastating.

I really didn't know what to do for a few months. I felt that I had let the previous generation of entrepreneurs down - that I had dropped the baton as it was being passed to me. I met with David Packard and Bob Noyce and tried to apologize for screwing up so badly. I was a very public failure, and I even thought about running away from the valley. But something slowly began to dawn on me – I still loved what I did. The turn of events at Apple had not changed that one bit. I had been rejected, but I was still in love. And so I decided to start over.

I didn't see it then, but it turned out that getting fired from Apple was the best thing that could have ever happened to me. The heaviness of being successful was replaced by the lightness of being a beginner again, less sure about everything. It freed me to enter one of the most creative periods of my life.

During the next five years, I started a company named NeXT, another company named Pixar, and fell in love with an amazing woman who would become my wife. Pixar went on to create the worlds first computer animated feature film, Toy Story, and is now the most successful animation studio in the world. In a remarkable turn of events, Apple bought NeXT, I retuned to Apple, and the technology we developed at NeXT is at the heart of Apple's current renaissance. And Laurene and I have a wonderful family together.

I'm pretty sure none of this would have happened if I hadn't been fired from Apple. It was awful tasting medicine, but I guess the patient needed it. Sometimes life hits you in the head with a brick. Don't lose faith. I'm convinced that the only thing that kept me going was that I loved what I did. You've got to find what you love. And that is as true for your work as it is for your lovers. Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work. And the only way to do great work is to love what you do. If you haven't found it yet, keep looking. Don't settle. As with all matters of the heart, you'll know when you find it. And, like any great relationship, it just gets better and better as the years roll on. So keep looking until you find it. Don't settle.

My third story is about death.

When I was 17, I read a quote that went something like: "If you live each day as if it was your last, someday you'll most certainly be right." It made an impression on me, and since then, for the past 33 years, I have looked in the mirror every morning and asked myself: "If today were the last day of my life, would I want to do what I am about to do today?" And whenever the answer has been "No" for too many days in a row, I know I need to change something.

Remembering that I'll be dead soon is the most important tool I've ever encountered to help me make the big choices in life. Because almost everything – all external expectations, all pride, all fear of embarrassment or failure - these things just fall away in the face of death, leaving only what is truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are already naked. There is no reason not to follow your heart.

About a year ago I was diagnosed with cancer. I had a scan at 7:30 in the morning, and it clearly showed a tumor on my pancreas. I didn't even know what a pancreas was. The doctors told me this was almost certainly a type of cancer that is incurable, and that I should expect to live no longer than three to six months. My doctor advised me to go home and get my affairs in order, which is doctor's code for prepare to die. It means to try to tell your kids everything you thought you'd have the next 10 years to tell them in just a few months. It means to make sure everything is buttoned up so that it will be as easy as possible for your family. It means to say your goodbyes.

I lived with that diagnosis all day. Later that evening I had a biopsy, where they stuck an endoscope down my throat, through my stomach and into my intestines, put a needle into my pancreas and got a few cells from the tumor. I was sedated, but my wife, who was there, told me that when they viewed the cells under a microscope the doctors started crying because it turned out to be a very rare form of pancreatic cancer that is curable with surgery. I had the surgery and I'm fine now.

This was the closest I've been to facing death, and I hope its the closest I get for a few more decades. Having lived through it, I can now say this to you with a bit more certainty than when death was a useful but purely intellectual concept:

No one wants to die. Even people who want to go to heaven don't want to die to get there. And yet death is the destination we all share. No one has ever escaped it. And that is as it should be, because Death is very likely the single best invention of Life. It is Life's change agent. It clears out the old to make way for the new. Right now the new is you, but someday not too long from now, you will gradually become the old and be cleared away. Sorry to be so dramatic, but it is quite true.

Your time is limited, so don't waste it living someone else's life. Don't be trapped by dogma - which is living with the results of other people's thinking. Don't let the noise of other's opinions drown out your own inner voice. And most important, have the courage to follow your heart and intuition. They somehow already know what you truly want to become. Everything else is secondary.

When I was young, there was an amazing publication called The Whole Earth Catalog, which was one of the bibles of my generation. It was created by a fellow named Stewart Brand not far from here in Menlo Park, and he brought it to life with his poetic touch. This was in the late 1960's, before personal computers and desktop publishing, so it was all made with typewriters, scissors, and polaroid cameras. It was sort of like Google in paperback form, 35 years before Google came along: it was idealistic, and overflowing with neat tools and great notions.

Stewart and his team put out several issues of The Whole Earth Catalog, and then when it had run its course, they put out a final issue. It was the mid-1970s, and I was your age. On the back cover of their final issue was a photograph of an early morning country road, the kind you might find yourself hitchhiking on if you were so adventurous. Beneath it were the words: "Stay Hungry. Stay Foolish." It was their farewell message as they signed off. Stay Hungry. Stay Foolish. And I have always wished that for myself. And now, as you graduate to begin anew, I wish that for you.

Stay Hungry. Stay Foolish.

Thank you all very much.