Zwabel’s Weblog

December 18, 2008

C++ Meta-Programming in KDevelop4: Another Milestone

Filed under: KDE,KDevelop — zwabel @ 5:12 am
Tags: , , , ,

Templates are one of the extremely powerful features of C++ that set it apart from most other languages. You can do most everything with them, from extremly complicated compile-time meta-programming like binary self-balanced trees, down to simple shared pointers or generic container classes.

When I switched to Linux a few years ago, I found KDevelop3 and liked it. However I noticed that it didn’t support code-completion for even the simplest C++ templates like for example a reference-counter pointer. That was my entry into KDevelop development. I implemented template support in KDevelop3 that had most of the needed features, and worked quite well when the needed libraries were processed. However the quality it could reach had some upper bounds, since there was no powerful backing structure like the DUChain, but rather just a flat completely string-based code-model.

Actually it was quite hard making the static DUChain structure compatible with dynamic templates, where a new declaration can be created with every use of a class. However I finally succeeded, and today I’ve reached a milestone that is far above of what KDevelop3, and probably even most other IDEs can reach: Full support for C++ template specialization instantiations.

Explicit Template Specializations
What basically motivated me to finish this was that even though the template support was already quite mature, the code-completion for __gnu_cxx::normal_iterator, the iterator used by std::vector, did not work correctly. The problem: It uses the template-class iterator_traits to compute its used types, which looks something like this:


template<class T>
struct Traits {
typedef T Type;
};

template<class T>
struct Traits<T*> {
typedef T Type;
};

This means that, depending one the template-parameter given, another class with different content is instantiated. This case is relatively easy, but this whole principle allows implementing most any algorithm statically, and thus can become arbitrarily complex. The good thing is that I already implemented a framework for the matching of implicit template-parameters to template-functions, that could partially be re-used to do the actual matching.

So the result of the last days is: Full code-completion for all stdc++ functions/classes that use iterator_traits, which is mainly the iterators, and well, also code-completion for any other C++ classes that use explicit specialization. And important: Without any hacks, it’s based on pure language understanding. 🙂 Technically this means that the internal C++ engine is now, apart from bugs and slowness, mostly feature-complete.

Meta Programming
As said above, explicit template specialization can not only be used to form new types, but also to compute numerical stuff statically. While this is not very useful in the form I’m going to present, it does serve as a utility to create really efficient code in practice. Today I’ve pushed the C++ engine in KDevelop4 so far, that it can actually evaluate such meta programs right within KDevelop. So here we go:
prime_meta
What you see on the screenshot is mainly a simple meta-program I found on the internet that can compute whether a number is a prime-number. I tuned it a little so the results can nicely be visualized graphically be KDevelop’s declaration highlighting: At the bottom, you see all the lines highlighted that were computed as prime numbers.

Useful Stuff
What I’ve presented until now is mainly backend features, that will be very helpful in practice without you noticing it, because they will make code-completion, use-building, etc. for more complex template-code “just work”.

There is also some annoying stuff about templates though. One of the main problem: They are not nice to write. You never know what types you’re actually dealing with. Even if the types are supposed to fit into some pattern, most IDEs cannot give you any code-completion while you’re writing a template class.

There is a feature in KDevelop4 to recover you from that misery though. So consider the following screenshot:
completion_within_template_1
You see a very simple template class called TemplateClass. Within its member function, you can not have any code-completion for the template type T, because it is unknown. However if you put the cursor above one of the instantiations at the bottom, in our case above “TemplateClass<Struct1>”, and push the shortcut to jump to its declaration, then KDevelop will remember that you’re interested in that specific instantiation, and will give you total correct code-completion for that instantiation within that class:
completion_within_template_2

Development Status
Apart from those features, I’ve worked a bit more on the backend, and fixed a lot of thread-safety and stability issues. This went so far that now, after a very long single-threaded time, I have enabled multi-threaded parsing again. This also leads to faster reparsing and re-highlighting while editing the document, because one parsing-thread is reserved only for parsing triggered by user-input.

Advertisement

December 10, 2008

Code-Navigation in KDevelop4: Meet the Magic Modifier!

Today I’m going to blog about the various duchain based navigation and browsing features available in KDevelop4. So let’s start right from the beginning. Consider you have just opened your project. Where to start? Graphically browsing project-trees is ok to get an overview, but it’s horribly inefficient if you need to do it to reach a place you know.

Quick Open
The Quick Open feature is the answer to that problem. KDevelop3 already had it, and IMO it was one of it’s most productivity-boosting features. Strangely it was/is not very well known among the users I talked to.

In short, the Quick Open feature is a simple list of all files, classes, and/or functions within your whole project. By typing into a line-edit you can filter that list.

I have re-implemented Quick Open for KDevelop4 in a much more flexible way, with several different data providers, probably similar to plasmas runner design. The data-providers are split up by “Scopes” and “Items”, allowing everything to be shown within the same list, or showing only specific parts. For the feature to work best, full-project parsing should be enabled, so classes/functions from within all open projects can be listed. Here you see it in action:
Quick Open

So now you’ve got a way to jump into arbitrary locations and get going. But what if you don’t want jump to an arbitrary location, but rather one in the same file? For this, there is an additional “Outline Quick Open”, based on the same framework, but only showing interesting items in the current file. The list can be filtered as well, but as additional gimmick the function/class you’re currently in is always pre-selected, which means you can comfortably use it repeatedly jump from one neighbour function to the next.

Local Navigation
When the duchain is working correctly, and the uses are built, nearly every symbol in a source-file is referencing a declaration in another place. That information is not only used for the highlighting, which will be the subject of another blog post, but also for direct navigation. Pushing CTRL+’.’ or CTRL+’,’ allows jumping directly to that declaration respectively definition.

One of the most useful local navigation features though is the following one. As you see in the following screenshot, all uses of the declaration under the cursor are automatically highlighted in yellow.
Highlighting
Since these highlighted uses are real “spots of interest”, it makes quite sense that the user might want to jump directly to one of them. So the shortcuts “Next Use” and “Previous Use” allow you directly cycling through them, jumping from one to the next, even across file boundaries. In my experience, since often your current interest is bound to a specific variable or function, this allows you getting where you want a lot faster.

The switch declaration/definition shortcut which allows jumping directly from within a definition to the related declaration and back, perfectly complements the team.

Navigation History
When it’s easy to jump to a random location, it should also be easy to get back. The code-browser implements an intelligent editor navigation history, that only inserts one history-entry for each function/class you’ve visited. Thanks to this, every really interesting place is in the history once, without too many useless jumps in between.

The Code Browser
In my opinion a really good C++ IDE should make a tool like Doxygen unnecessary. The IDE should know all the relevant information, it can update it dynamically, and it can present it to the user precisely where needed. That’s why I implemented the so called navigation-widget right when I started working on KDevelop4 C++ code completion. It is a html text-view containing some nicely formatted text, similar to the output of doxygen. The widget matured together with the duchain, being able to show more and more useful information with time, and at some point it was far enough to be really useful. In the end it found its permanent home in the Code Browser, visible at bottom in the following screenshot:
Code Browser
Listing all the information this widget shows would not fit into this blog post. It really shows nearly anything interesting about the shown item, and especially it allows jumping to related items from return-types, template-parameters, inheriters, base-classes, overriders, it allows showing the preprocessed text of macro invocations, showing all declarations from include files, jumping to direct source-locations, searching uses, and so on. When full-project parsing is enabled, this mostly surpasses the quality of what doxygen shows, since it can deal with C++ specifics like templates or macros really well.

The Code Browser always shows informations about the item currently under the mouse- or text-cursor in the document, unless the little lock at the top right is activated, which preserves the view.

Since a real developer hates using the mouse when in hacking mode, the navigation widget is completely keyboard accessible.

The Magic Modifier
Now thanks to the navigation widget, we do have a lot of keyboard-accessible information to show. Wouldn’t it be great if exactly that information was available from everywhere, with a consistent accessibility scheme? This is where the Magic Modifier(A marketing-name invented by me 😉 ) comes into play. The Magic Modifier allows you peeking from the sketchy code-realm into the semantic one, seeing the connections behind the code. Currently it is hard-coded to the “ALT” key, so that’s actually all you need to remember. This is how it works:
– By pressing the ALT key, you will in some way get presented a navigation-widget. While you keep it pressed, you can navigate within it using the arrow-keys and the enter-key. Once you release ALT again, it will again disappear, unless you push and release it really fast, which equals a “toggle”.
Completion Expanded
On this screenshot you see how the first item in the completion-list was expanded because ALT is pressed. The same thing is possible within all quickopen lists. This also works directly within the editor. When the code-browser is visible, you can always navigate within it using ALT+Arrows. When the code-browser is not visible and ALT is pressed, the following keyboard-accessible tooltip is popped up, and disappears once ALT is released again.
Tooltip

So thanks to the Magic Modifier, you’re always just a single keypress away from the semantic information behind your code.

Html-Like Browsing
Sometimes when not in hacking-mode, it is comfortable to browse through the code comfortably without any keyboard-interaction. One way to achieve this is by using the new navigation tooltip I have added the last days, or the code-browser, by first pointing at the interesting item. However due to the duchain structure, the complete document could be interpreted as a single hyperlinked document, where each use is a link to its declaration.

While pressing either CTRL or the Magic Modifier, you can click onto uses directly in the document, and it will bring you directly to the used declaration. The mouse-cursor will change to a hyperlink-style pointing hand when this is possible. For even more comfort without any keyboard interaction, it’s possible to go into a permanent “browsing” state, by clicking the little green lamp toolbutton in the code browser.

So now you have a lot of navigation tools at hand, which all together allow really efficient working and browsing. I recomment every KDevelop user to memorize all of them, so you have the right tool at Hand in the right moment.

Development Status
In the last weeks I’ve once again spent quite some time on the duchain backend. Parsing and especially update-checking is a lot more efficient now, somewhat less disk-space is consumed, several consistency and crash problems have been fixed.

Also tons of bugs in code-completion and general C++ support were fixed, the shown items in the completion-list are now intelligently shortened when they are too long by leaving away template-parameters. The completion-list in general looks better now with less colors and lighter icons, I’ve implemented the consistent “Magic Modifier” navigation, and tons of other stuff that I don’t remember.

Just by the way if you’re wondering why my development speed is so fast at the moment: Soon I will begin working on my Diploma Thesis, which will leave a lot less time for KDevelop. I will be working on a C++ project though, and of course I will be using KDevelop4, which 1. means that I will definitely not lose KDevelop out of focus, and which 2. means that I want KDevelop to be in a good state until then.

What makes me a bit sad is that the rest of KDevelop is not really taking off. Debugger, project-management, documentation-integration, etc. all still need quite some work before a 4.0 release. One interesting thing is that Aleix Pol Gonzales has been improving the cmake duchain support, which means that many of the described features will also be available while working on cmake files at some point.

Blog at WordPress.com.