Zwabel’s Weblog

February 4, 2009

KDevelop4: Automatic include-directives and forward-declarations

Filed under: KDE,KDevelop — zwabel @ 1:09 am
Tags: , , , , , ,

Missing Include Completion
C++ is a great and powerful programming-language. Yet it has the downside against some other languages, that you always have to deal with include-directives or forward-declarations before you can use a class.

This is a factor that often motivates me not to create too many different source-files, although design-wise that would make sense. Wouldn’t it be nice if you could just start hacking without caring about the whole visibility stuff? Especially when you have to do the same includes again and again, and already know the library you’re using and it’s classes, this is nothing more than an annoyance.

I had implemented a feature to automatically add include-directives within KDevelop 4 about a year ago, but it wasn’t as comfortable and useful as it could be. During the last days, I’ve taken the time to polish up this feature so it’s worth a blog-post.

The whole thing is based on the DUChain, and it respects all declarations from within the global duchain store. This means that from the moment on that KDevelop has processed a source-file, you can start just using the declarations from within that file from anywhere. Whenever you type something and the completion-list does not offer any completions, which means it’s not visible, then the missing-include completion will start searching the duchain store for matching declarations, and when it finds some, it will offer you right within the completion-list to automatically add an include-directive for you.

This works in many different situations: If you try calling a function or constructing a class, template-instantiate a class, if you try to access the contents of a class that is either unresolved or has only been forward-declared, or if you just type in the name of an arbitrary declaration.

Sometimes however when writing header files, you do not need the whole class-definition but rather just a forward-declaration. From today on, the very same list will show you an entry to automatically add a forward-declaration for the typed class for you. This even works with template-classes, and it correctly respects namespaces, creating namespace declarations around the forward-declarations as needed.

Together, under the circumstance that KDevelop already knows the classes you are working with, this frees you of one annoying part of the hacking.

Examples

This is what happens when you try calling a known function that has not been included yet:
missing_include_completion_function_call

And this is the effect you will have when you push the up-arrow and execute the “Add #include …” action:
missing_include_completion_function_call_result

This is what happens when you simply type the name of a known class:
missing_include_normal_completion1

And this is what you’ll get when you pick the “Add forward-declaration” action. A fully valid forward-declaration was inserted. Well, apart from the fact that std::allocator is not defined. 😉
missing_include_normal_completion_result

Now std::vector is forward-declared. But what if you actually try using it? KDevelop will notice it, and will offer you including the correct header for being able to use it:
missing_include_member_completion

And this is what happens when you try to instantiate a known but not included template. Notice that it does offer to include the forwarding-header “QMap”, and not only the header that really contains the class which is “qmap.h”:
missing_include_template_completion

And now a little demonstration how this can help you using most any library:
icore_1

See that ICore has a lot of members that again return pointers to other classes, that of course were only forward-declared. This usually means that before you can use the result of activeSession(), you have to find out where KDevelop::ISession is defined, and include it manually:
icore_2

With KDevelop4, the content is only one key-press away:
icore_3

Disclaimer
This is a feature for experienced users who know what they are doing. It always has to be reviewed, and KDevelop does NOT do any programming for you. 😉

Development Update
Although I don’t have that much time any more, I did find enough time to do quite some polishing during the last weeks. Many crashes and bugs were fixed, and I even found time to imlement some smaller features.
Here the most important points:
– The implementation-helper and signal/slot completions now are shown within separate groups in the completion-list, that are shown in an appropriate place(usually right at the top)
– Access-rights is now fully respected by the completion-list, including friend-declarations.
– Not found declarations can be highlighted with an error-underline now(There’s a new “highlight semantic problems” option in the configuration)
– Added context-sensitive code-completion for builtin C++ keywords
– Full code-completion, use-building and refactoring for namespace names
– Less annoying automatic completion, because it auto-hides when an item in the list is matched
– Workaround a multi-threading problem within kdelibs and ksycoca that caused KDevelop to crash very often
– Fix a repository-locking problem that made kdevelop reproducably crash under special circumstances

Although we do not have all the features we want together, we will be releasing a beta-version of KDevelop4 soon, because we believe it’s already a useful application. Aleix Pol Gonzales has started working on the much needed documentation integration, and Hamish Rodda has today started some new discussion on the debugger, and is planning to bring it into a usable state.

December 18, 2008

C++ Meta-Programming in KDevelop4: Another Milestone

Filed under: KDE,KDevelop — zwabel @ 5:12 am
Tags: , , , ,

Templates are one of the extremely powerful features of C++ that set it apart from most other languages. You can do most everything with them, from extremly complicated compile-time meta-programming like binary self-balanced trees, down to simple shared pointers or generic container classes.

When I switched to Linux a few years ago, I found KDevelop3 and liked it. However I noticed that it didn’t support code-completion for even the simplest C++ templates like for example a reference-counter pointer. That was my entry into KDevelop development. I implemented template support in KDevelop3 that had most of the needed features, and worked quite well when the needed libraries were processed. However the quality it could reach had some upper bounds, since there was no powerful backing structure like the DUChain, but rather just a flat completely string-based code-model.

Actually it was quite hard making the static DUChain structure compatible with dynamic templates, where a new declaration can be created with every use of a class. However I finally succeeded, and today I’ve reached a milestone that is far above of what KDevelop3, and probably even most other IDEs can reach: Full support for C++ template specialization instantiations.

Explicit Template Specializations
What basically motivated me to finish this was that even though the template support was already quite mature, the code-completion for __gnu_cxx::normal_iterator, the iterator used by std::vector, did not work correctly. The problem: It uses the template-class iterator_traits to compute its used types, which looks something like this:


template<class T>
struct Traits {
typedef T Type;
};

template<class T>
struct Traits<T*> {
typedef T Type;
};

This means that, depending one the template-parameter given, another class with different content is instantiated. This case is relatively easy, but this whole principle allows implementing most any algorithm statically, and thus can become arbitrarily complex. The good thing is that I already implemented a framework for the matching of implicit template-parameters to template-functions, that could partially be re-used to do the actual matching.

So the result of the last days is: Full code-completion for all stdc++ functions/classes that use iterator_traits, which is mainly the iterators, and well, also code-completion for any other C++ classes that use explicit specialization. And important: Without any hacks, it’s based on pure language understanding. 🙂 Technically this means that the internal C++ engine is now, apart from bugs and slowness, mostly feature-complete.

Meta Programming
As said above, explicit template specialization can not only be used to form new types, but also to compute numerical stuff statically. While this is not very useful in the form I’m going to present, it does serve as a utility to create really efficient code in practice. Today I’ve pushed the C++ engine in KDevelop4 so far, that it can actually evaluate such meta programs right within KDevelop. So here we go:
prime_meta
What you see on the screenshot is mainly a simple meta-program I found on the internet that can compute whether a number is a prime-number. I tuned it a little so the results can nicely be visualized graphically be KDevelop’s declaration highlighting: At the bottom, you see all the lines highlighted that were computed as prime numbers.

Useful Stuff
What I’ve presented until now is mainly backend features, that will be very helpful in practice without you noticing it, because they will make code-completion, use-building, etc. for more complex template-code “just work”.

There is also some annoying stuff about templates though. One of the main problem: They are not nice to write. You never know what types you’re actually dealing with. Even if the types are supposed to fit into some pattern, most IDEs cannot give you any code-completion while you’re writing a template class.

There is a feature in KDevelop4 to recover you from that misery though. So consider the following screenshot:
completion_within_template_1
You see a very simple template class called TemplateClass. Within its member function, you can not have any code-completion for the template type T, because it is unknown. However if you put the cursor above one of the instantiations at the bottom, in our case above “TemplateClass<Struct1>”, and push the shortcut to jump to its declaration, then KDevelop will remember that you’re interested in that specific instantiation, and will give you total correct code-completion for that instantiation within that class:
completion_within_template_2

Development Status
Apart from those features, I’ve worked a bit more on the backend, and fixed a lot of thread-safety and stability issues. This went so far that now, after a very long single-threaded time, I have enabled multi-threaded parsing again. This also leads to faster reparsing and re-highlighting while editing the document, because one parsing-thread is reserved only for parsing triggered by user-input.

December 10, 2008

Code-Navigation in KDevelop4: Meet the Magic Modifier!

Today I’m going to blog about the various duchain based navigation and browsing features available in KDevelop4. So let’s start right from the beginning. Consider you have just opened your project. Where to start? Graphically browsing project-trees is ok to get an overview, but it’s horribly inefficient if you need to do it to reach a place you know.

Quick Open
The Quick Open feature is the answer to that problem. KDevelop3 already had it, and IMO it was one of it’s most productivity-boosting features. Strangely it was/is not very well known among the users I talked to.

In short, the Quick Open feature is a simple list of all files, classes, and/or functions within your whole project. By typing into a line-edit you can filter that list.

I have re-implemented Quick Open for KDevelop4 in a much more flexible way, with several different data providers, probably similar to plasmas runner design. The data-providers are split up by “Scopes” and “Items”, allowing everything to be shown within the same list, or showing only specific parts. For the feature to work best, full-project parsing should be enabled, so classes/functions from within all open projects can be listed. Here you see it in action:
Quick Open

So now you’ve got a way to jump into arbitrary locations and get going. But what if you don’t want jump to an arbitrary location, but rather one in the same file? For this, there is an additional “Outline Quick Open”, based on the same framework, but only showing interesting items in the current file. The list can be filtered as well, but as additional gimmick the function/class you’re currently in is always pre-selected, which means you can comfortably use it repeatedly jump from one neighbour function to the next.

Local Navigation
When the duchain is working correctly, and the uses are built, nearly every symbol in a source-file is referencing a declaration in another place. That information is not only used for the highlighting, which will be the subject of another blog post, but also for direct navigation. Pushing CTRL+’.’ or CTRL+’,’ allows jumping directly to that declaration respectively definition.

One of the most useful local navigation features though is the following one. As you see in the following screenshot, all uses of the declaration under the cursor are automatically highlighted in yellow.
Highlighting
Since these highlighted uses are real “spots of interest”, it makes quite sense that the user might want to jump directly to one of them. So the shortcuts “Next Use” and “Previous Use” allow you directly cycling through them, jumping from one to the next, even across file boundaries. In my experience, since often your current interest is bound to a specific variable or function, this allows you getting where you want a lot faster.

The switch declaration/definition shortcut which allows jumping directly from within a definition to the related declaration and back, perfectly complements the team.

Navigation History
When it’s easy to jump to a random location, it should also be easy to get back. The code-browser implements an intelligent editor navigation history, that only inserts one history-entry for each function/class you’ve visited. Thanks to this, every really interesting place is in the history once, without too many useless jumps in between.

The Code Browser
In my opinion a really good C++ IDE should make a tool like Doxygen unnecessary. The IDE should know all the relevant information, it can update it dynamically, and it can present it to the user precisely where needed. That’s why I implemented the so called navigation-widget right when I started working on KDevelop4 C++ code completion. It is a html text-view containing some nicely formatted text, similar to the output of doxygen. The widget matured together with the duchain, being able to show more and more useful information with time, and at some point it was far enough to be really useful. In the end it found its permanent home in the Code Browser, visible at bottom in the following screenshot:
Code Browser
Listing all the information this widget shows would not fit into this blog post. It really shows nearly anything interesting about the shown item, and especially it allows jumping to related items from return-types, template-parameters, inheriters, base-classes, overriders, it allows showing the preprocessed text of macro invocations, showing all declarations from include files, jumping to direct source-locations, searching uses, and so on. When full-project parsing is enabled, this mostly surpasses the quality of what doxygen shows, since it can deal with C++ specifics like templates or macros really well.

The Code Browser always shows informations about the item currently under the mouse- or text-cursor in the document, unless the little lock at the top right is activated, which preserves the view.

Since a real developer hates using the mouse when in hacking mode, the navigation widget is completely keyboard accessible.

The Magic Modifier
Now thanks to the navigation widget, we do have a lot of keyboard-accessible information to show. Wouldn’t it be great if exactly that information was available from everywhere, with a consistent accessibility scheme? This is where the Magic Modifier(A marketing-name invented by me 😉 ) comes into play. The Magic Modifier allows you peeking from the sketchy code-realm into the semantic one, seeing the connections behind the code. Currently it is hard-coded to the “ALT” key, so that’s actually all you need to remember. This is how it works:
– By pressing the ALT key, you will in some way get presented a navigation-widget. While you keep it pressed, you can navigate within it using the arrow-keys and the enter-key. Once you release ALT again, it will again disappear, unless you push and release it really fast, which equals a “toggle”.
Completion Expanded
On this screenshot you see how the first item in the completion-list was expanded because ALT is pressed. The same thing is possible within all quickopen lists. This also works directly within the editor. When the code-browser is visible, you can always navigate within it using ALT+Arrows. When the code-browser is not visible and ALT is pressed, the following keyboard-accessible tooltip is popped up, and disappears once ALT is released again.
Tooltip

So thanks to the Magic Modifier, you’re always just a single keypress away from the semantic information behind your code.

Html-Like Browsing
Sometimes when not in hacking-mode, it is comfortable to browse through the code comfortably without any keyboard-interaction. One way to achieve this is by using the new navigation tooltip I have added the last days, or the code-browser, by first pointing at the interesting item. However due to the duchain structure, the complete document could be interpreted as a single hyperlinked document, where each use is a link to its declaration.

While pressing either CTRL or the Magic Modifier, you can click onto uses directly in the document, and it will bring you directly to the used declaration. The mouse-cursor will change to a hyperlink-style pointing hand when this is possible. For even more comfort without any keyboard interaction, it’s possible to go into a permanent “browsing” state, by clicking the little green lamp toolbutton in the code browser.

So now you have a lot of navigation tools at hand, which all together allow really efficient working and browsing. I recomment every KDevelop user to memorize all of them, so you have the right tool at Hand in the right moment.

Development Status
In the last weeks I’ve once again spent quite some time on the duchain backend. Parsing and especially update-checking is a lot more efficient now, somewhat less disk-space is consumed, several consistency and crash problems have been fixed.

Also tons of bugs in code-completion and general C++ support were fixed, the shown items in the completion-list are now intelligently shortened when they are too long by leaving away template-parameters. The completion-list in general looks better now with less colors and lighter icons, I’ve implemented the consistent “Magic Modifier” navigation, and tons of other stuff that I don’t remember.

Just by the way if you’re wondering why my development speed is so fast at the moment: Soon I will begin working on my Diploma Thesis, which will leave a lot less time for KDevelop. I will be working on a C++ project though, and of course I will be using KDevelop4, which 1. means that I will definitely not lose KDevelop out of focus, and which 2. means that I want KDevelop to be in a good state until then.

What makes me a bit sad is that the rest of KDevelop is not really taking off. Debugger, project-management, documentation-integration, etc. all still need quite some work before a 4.0 release. One interesting thing is that Aleix Pol Gonzales has been improving the cmake duchain support, which means that many of the described features will also be available while working on cmake files at some point.

Blog at WordPress.com.