Synopsis

Umit is a nmap frontend started with the sponsoring of Google during the Summer of Code 2005 and improved during Summer of Code 2006. Like any other young tool, Umit needs some adjustments and improvements to fulfill the needs of it's users. After the release of it's last version, the project received loads of feedbacks from the community, pointing out and suggesting new features.
The purpose of this mentoring proposal is to create new tools to integrate Umit and make network admins life's easier.

How to make a proposal

Last week I made a speech at Universidade Estadual de Goias about open source and Summer of Code. During the speech I talked about how the proposal should be made, and now, I'm making the presentation available in Brazilian Portuguese and in English to help students on their duties of creating a good proposal to increase their chances of been accepted this year.

Ideas list

Plus

Some work that I've been developing or planning to develop soon, independently of the Summer of Code.

UmitMapper

After analising fe3d, Cheops and NCartographer, I thought about how I should implement this feature at Umit in a way that it won't be a processor resource eater, like fe3d, or visually limited as Cheops and NCartographer. Another preoccupation was to don't add more dependencies. I used them (fe3d, Cheops and Ncartographer) and analised their behaviors and solutions to take a better decision for the Umit Mapper problem.

The conclusion was that user needs more than a mapper: user needs a useful mapper that they can use with a few clicks to browse the network in the easiest way without the need of painfully saving scan results and openning them in another application just to browser it. The Umit project's duty is to make users lifes easier, helpping them to do their jobs faster, and here is where UmitMapper comes in help.

fe3d
This software use the nmap XML output to generate 3D graphics of the scanned network. The visual efect is amazing. I confess that my first thought was about doing something similar on Umit. However, analysing this paradigm, I realised that fly across a 3D model isn't much convenient, mainly when the model is representing a huge network. Because of it's low usability, this model won't atract users, even with the cool visual efect provided by a 3D model that we can show to our friends and say "all your networks are belong to us".

Thinking about the needs of a network administrator, I perceived that what he really needs is a way to have a global visualization of the network and it's topology. Another thing I noticed is that, generally, administrators have the interest on navigate through the topology to see selected informations and details about each host without getting lost. This feature unfortunatelly isn't provided by fe3d, making it kind of useless for the real network administrator that needs to save time navigating the network and hosts informations.

Cheops/Cheops-NG
Despite the not that cool and amazing visual efect owned by fe3d, Cheops provides more functionality and usability. Many of it's current features is already provided by nmap (as I could see, it doesn't uses nmap) and lots of it's interface features are at Umit. However, talking about it's mapper, when it comes to a huge network it becomes harder to navigate through it. Small networks are well attended by Cheops, but our goal is to attend well networks of any size.

NCartographer
One of the most recent tool developed for this purpose, made on last Summer of Code, is also intended for an overview of the network found in the scan result. Yet, as it is not integrated with nmap, you must make the scan in separate, save the result and them open it at NCartographer. The result view also doesn't scale well with large scan results, and we need a solution for networks of any size.

Umit Mapper
The Umit Mapper goal is to provide a network topology diagram generation as well as an ease way to navigate through it for a quick analysis without getting lost in a huge network. This way, user can navigate through the informations of hosts from each network without getting confused. Umit Mapper is going to be developed in PyGTK, and will be integrated to the Umit interface. At first sight it can feel frustrating the fact that the mapper is going to be developed in PyGTK, but there are many advantages:

As usual, I developed a visual prototype for the Umit Mapper. It doesn't have any functionality yet. It was developed just to serve as a basis for it's design. At the window shown by the screenshot below, user will be able to have a global view of the network and it's topology. The blue globe is been used as a network icon, and the blue square with arrows means a router. However, these icons and the window design are temporary. The accepted student will have to improve the interface appearence and make a better design for it.

Higher network topology view (with scanned networks)

By clicking on any network icon, user can see the hosts that belongs to that network. The following screenshot is a simplified visualization of the scanned hosts, showing their respective ips and operating systems icons (if recognized) below the icon and the address of the current network. This network address and icon is intended to let user know where he is all the time, avoiding user to get confused or lost about where he is inside the topology. If any host icon is clicked, is shown in the right side of the window informations about the host, like openned ports and it's vulnerability level. Clicking on the network icon, gets user back to the screenshot above.

Simple visualization of hosts from a selected network

The next screenshot is another visualization option that shows the openned ports of each host below it's icon. In this view, users won't need to click over each icon to realise which ports are openned making it easier to have a global view of the network and the services provided on it.

Another visualization of hosts from a selected network

With the model of the next screenshot, it's possible to easily navigate through the network listing hosts informations individually and with an attractive visual.

Individual visualization of hosts from a selected network

User will be able to navigate through the result using filters, that group scanned hosts by a selected common attribute like openned port, vulnerability level or operating system.

Filtering hosts by open ports, vulnerability level and operating system

Umit Mapper is intended to be more than a mapper and provide features really useful for it's users with higher usability.

Despite the mockups and the ideas I put here, the accepted student will have the freedom to suggest more ideas, discuss and argue about the proposal and some statements I made. This proposal is only the basics to help them creating their own proposals.

Some requirements that students must pay attention while making their proposals:

UmitWeb

The goal of this proposal is to create a web interface able to reproduce the main Umit interface functionalities. This interface is intended to make network admins lifes easier, by providing a tool that they can use to make remote scans from anywhere in the globe, using just a browser, and having the scan result in their mail or even in the browser as a report.

Here is why users need a web interface: sometimes you just want to know what is happening with your network without the need of going inside it using a VPN or an SSH to run a command line scan or call the Umit interface remotelly. Sometimes, you're not at home or with your laptop and you need to keep your eyes in your network using your grandma's computer without the need of installing anything there and yet having the same level of functionalities you would have by using the standard interface.

Some requirements that students must pay attention while making their proposals:

Nmap Wrapper for Python

The wrapper intention is to provide a module from which you can create an Nmap instance, set the desired options and targets and run it without the need of executing it in another process. The wrapper must allow access of Nmap funcionalities like estimated time to finish the scan and runtime user interaction.

User's almost won't note any change with this wrapper. This is a feature that is going to make Nmap and Umit developers life's easier, while adding new features or even integrating with future Nmap options and functionalities.

NSE Facilitator

The idea of the NSE facilitator is to provide a better integration of Umit, Nmap and NSE in such a way that even a newbie could easilly run and share scripts with a few clicks, instead of finding them in the web, downloading, reading documentation to know how to use it with Nmap and running a long command to be able to use it.

Some requirements that students must pay attention while making their proposals:

Wizard and Profile interface designer

Umit's command wizard and profile editor were designed to be created in runtime, based on an XML definition, which says the interface options and their arrengements in the GUI. Altought this XML file is very easy to understand and modify, it is not the best way of making the user's life easier. These editors were designed to allow user to add or remove any option he feels like, or even add the newest nmap option that is not covered by the current Umit version yet. Editing the XML file, user can even combine options and make them easier to use during command or profile creations. The interface designer is a program that would let user add, remove or edit the profile and wizard interface in a visual way and without the need of hand editing the XML file.

Some requirements that students must pay attention while making their proposals:

Independent Features

Besides projects like UmitMapper and UmitWeb, there are some Umit improvement ideas that students could work for. Although these are feature development ideas, they're not less important that the independent projects listed above. Umit needs to grow in every directions, mainly in usability directions, because it's main purpose it's to ease some every-day tasks performed by network admins.

Students with a good background in Python, GTK and usability are elegible to work on these features during the summer. Here follows a list of some of the features that students can invest their time on:

Non-Root user limitations

As known, non-root users has limited options on nmap. Currently, this matter is not well supported on Umit, and when a non-root user tries to use a root option, Umit behave strangely. Sometimes it show a error msg, but sometimes it doesn't. I'm planning on making Umit avoid the use of root options by non-root users. Umit will also have a better support for nmap non-root error messages in case user insert by hand those root options at the command field. This will make Umit more user-resistent.

Command line argumments

Umit should be able to receive command line arguments, so user's could be able to call umit with arguments specifying that it should run a scan profile after it is openned, for example, and users that use command line, would feel more motivated to use umit because they won't need to run one command to open the interface, waits until it is completly openned and them type a command to be executed or choose a profile. These seconds are too much for a network admin and he surely wants to save them by doing everything he wants in the command line, and them just go after Umit to see the results in a more confortable way.

Send results by e-mail

This feature is responsible for sending a formated result to user's e-mail, with the normal and XML output attached. As this feature is going to be used by other projects, like the Network inventory, Scan Scheduler and UmitWeb, it must be developed inside umitCore module, so it can be re-used by other projects.

Improve profile editor

Currently, the profile creating/editing/removing progress is not intuitive. This issue ends with usability problems, and must be fixed quickly. Yet, the profile editor interface itself can be better designed and arranged to make user's life even easier.

Nmap runtime interaction

Nmap has some runtime interaction shortcuts that users can call to increase/decrease verbosity of the output, turn on/off packet tracing or even get the elapsed and estimed time to finish of the scan. But, currently, these options can only be used while running Nmap from command line. The idea, is to let user call these options from Umit interface, and maybe create some widgets to ease the task of using such options.

Progress Bar

A progress bar to show the scan progress to users. This is specially usefull for scans that takes too long to finish. The progress bar can make use of the Nmap's runtime interaction option which shows the elapsed time and the estimated time to finish of the scan.

Network Inventory

The Network Inventory is intended to help users to easily keep track of the hosts on his network. This is the way this feature is going to work: User enter with basic informations about a regular scan (targets, nmap scan options, etc) and the network inventory system will take care of execute the scan regularly to check changes on the network, in a frequency defined by the user. If any relevant change is noted since the last network scanning, user is notified about this change (through the Umit interface or by e-mail). Let's consider the hypothesis that a machine was violated and the intruder left an active backdoor running and, after that, he left the machine withou been noted. When the Network Inventory scan the network again, it will perceive the new openned door at this machine and notice the network administrator. With this new Umit feature, the administrator can have more chances to detect this kind of trouble in time to avoid more losses.

Another feature related to the Network Inventory is the history. Each verification scan made by the inventory is logged at the history. Using the history, the network administrator will be able to verify previous states of the network.
The maximum amount of logs that is going to be stored can be defined by the user.

Inventory is associated with another Umit feature that is going to be described ahead in this proposal. There is only one dependency related to this new feature, that is the SQLite. However, SQLite now is part of the python's 2.5 default library and won't be a dependency anymore for those who is going to use python 2.5. Anyway, SQLite is a pretty small library and is very easy to install.

Quick connect

Sometimes, the user scan the network searching for hosts serving on a given port. Later, with this information, he usually tries to connect to this host using a client application. Currently, a Umit's user must scan the network through the Umit interface and mannually execute an external client to connect to the host. This new feature will allow user to connect to the host using his preferred client directly from Umit's interface by clicking on a quick-button or a right-click over the host.

Let's suppose that you has just scanned and verified that a given host is serving FTP, and now, you want to establish a FTP connection to this host. With a single click over the quick-connection button, or a right-click over the host, Umit will try to execute the default FTP client application providing the host and port information for the connection. If there's no default FTP client application registered, Umit will ask user to provide information about which application should be executed for this kind of service. This asking phase will happen only once, on the first attempt to use this feature to connect on the selected service. The informations provided by the user are going to be stored for future use, and can be easily edited whenever the user feel like it.

Some examples of common scanned services that user usually tries to connect after the scanning:

This feature is very useful because is going to ease the everyday's scan-connect task, and it won't add any dependency.

Scan Scheduler

In the end of a speaking about Umit, I was questioned if Umit had a Scan Scheduler. At first sight, I felt strange about the question, but soon I realised it's importance. The Scan Scheduler will be useful to schedule scans and to support the network inventory. As a prototype, I developed a cron like parsing module for the Scan Scheduler and a daemon to verify if something is scheduled to be executed in the moment. You can see these modules here:

Despite they're only prototypes they already works pretty fine, and minor adjusts will be needed. The Scheduler will run as a daemon and user will be able to start it from Umit interface or at operating system initialization. This daemon responsibility is to take care of scan and network inventory schedules, with the duty of verify relevant changes and notify user by e-mail. This feature won't add any dependency to Umit.

Some requirements that students must pay attention while making their proposals:

Plus

Usability Test Suite

Usability must be matured, and the Umit goal is to be always a step forward on this subject. During the development of the proposed features for this SoC, I'll be preoccupated on a better adequacy of the interface for the user daily needs, increasing keyboard short-cuts and grabbing informations on how users use Umit and how they behave while working with it. These informations are useful to place widgets in better places, define shortcuts, add features, etc. To grab these informations, I developed a module called usabilityTest, which is a suite that essentially does the following: shows a form to grab user experience information, starts umit, video-recording and key logging user's desktop and asking him to accomplish some tasks at Umit. Here follows some screenshots:

User form that collects some user personal and experience informations. These informations are intended to help us understand user behaviors.

Yet the user form that collects some user personal and experience informations. In this particular form, user will say which experience does he have with networks, scanning tools, frontends, etc.

The usability test suite was develop in a way that it can be used by any other project, because the user information form and the tasks at the task manager are defined in a XML file. This way, other projects can change the user information form questions and the tasks on task manager to better fit their needs and easily grabbing important informations from user behavior.

Currently, it uses xvidcap to record the screen and lkl as a keylogger. Although the Task Manager is almost done, it is possible to run and test it without any problem. The usability test suite is been kept inside Umit repository, inside the usability directory, but this suite can turn into a separated open source project in the future.

Usability test phase: Umit interface and the Task Manager window. Everything user does here is video-recorded and the keys are logged.

Here is a sample video that I made testing the usbility test suite:

The video goes pretty fast because the machine on which I ran the test is very slow, and the video recording tool I used (xvidcap) wasn't able to capture the minimum number of frames to make a smooth video. As this video was only a test to see the result of the usability test suite execution, we don't need to worry about the video speed yet.

Dogtail

Some automated tests are also going to be made for Umit GUI using Dogtail. The idea is to make sure about the sanity of the GUI behavior and make sure that it is able to handle every Nmap option and expression without raising any exception.

Mac OS X Installer

Umit already works on Mac OS X, but it's very hard to install it from source there. As we don't want to botter user that much, I'm planning on making a installation package (.app) for Mac OS X, and spread Umit among OSX users.

Umit porting and installers for Maemo

From Maemo oficial web site:

"What is maemo?
Maemo is an open source development platform to create applications for Nokia Internet Tablet products like Nokia N800 and Nokia 770. The platform gives developers a powerful Linux based development environment and optimized end-user interface for handhelds."


Maemo already have a nmap package for it, and as it is used to access internet and networks wirelessly, Umit can be a great tool to have on it. It already runs on it, as we can see in the pics bellow, but it needs some interface improvements.

A porting of Umit for Maemo would include using Hildon to adapt the GTK interface appearence and behaviors and a package to ease the task of installing it. As it is a portable device, it's really boring to open a terminal, get into the directory of the package, un-tar it and type: "python setup.py install". There is a package system based on apt-get for Maemo, and we shaw use it to make things easier.

Screenshot of Umit splash on Gustavo's Nokia 770

Camshot of Umit splash on Gustavo's Nokia 770

New web site

Currently, the Umit web site is not good enough for an open source project. It doesn't have a section for documentation, downloads, about, etc. And the design looks ugly. I'll be working on it while students do their works. It's also a good opportunity to make sections in the website to hold their projects, and respectives documentations as well.