Zwabel’s Weblog

January 8, 2009

C++ IDE Evolution: From Syntax Highlighting to Semantic Highlighting

Filed under: KDE,KDevelop — zwabel @ 3:19 am

Most of us developers are so acccustomed to syntax highlighting, that we couldn’t live without it. Within the last years, it happened to me a few times that I had to look at C++ code with an editor that does not have it. Every single time my initial feeling was that I was looking at a large unstructured text-blob, totally unreadable. Putting some additional cognitional energy into the task, I was able to solve the problem in the end, but there is no doubt that syntax highlighting does increase productivity, it is not just eyecandy.

Syntax Highlighting
Now the first interesting question is: What exactly is it about syntax highlighting that makes the text easier to work with?

When trying to understand what the code does, we usually first try to recognize its coarse structure. For that, we need a fast overview of the code. With that overview, we can decide where we want to continue concentrating on. Now the problem is, while building this overview, really many words have to be scanend. Actually reading all of them would take a long time, and would be very annoying. Highlighting specific words in deterministic colors helps us reducing that load, by giving us familiar orientation points and patterns that our eyes can “hook” on, and allows us finding the specific position we’re searching faster.

So syntax highlighting helps us keeping an overview or finding the place we’re searching for. However it can _not_ help us actually understanding the code, because by the pure definition of “syntax”, it can only highlight by what the code looks like, not by what the code means, since that requires wider knowledge.

Semantic Highlighting
To overcome that limitation, deeper knowlege of the code is required. Right from the beginning, the DUChain in KDevelop was designed to represent exactly that knowledge, and more advanced code-highlighting was one of the basic motivation points behind creating it at all.

So how can it help? There is a few points to this:
1. Additional structure. Due to a now much wider applied highlighting, the code has a lot more colorful structure, which might lead to the same benefits syntax highlighting in general brings. However this is arguable. To some, the additional structure might even seem chaotic, since it’s just too much structure for them. It is definitely something you need to get used to.

Example: Look at this piece of code without semantic highlighting. It optically contains 2 big blobs of code, that to me, already being used to semantic highlighting, look quite unreadable.
semantic_highlighting3
Now look at the same thing with semantic highlighting. The additional structure splits the code-blobs up, and makes them perfectly readable.
semantic_highlighting2

2. Recognizing errors: When specific elements are always colorized in the same way, for example global items, enumerators, items in the local class, etc., then you will at some point expect them to be highlighted that way, and you will notice errors much earlier in cases the highlighting conflicts what you expect.
Example: See all the items beginning with “m_”, they are highlighted in brown. All class-local items are highlighted in that color. If for example m_quickOpenDataProvider was actually a global object, then it would be highlighted differently, and you’d notice the problem right away(This is most useful with function-calls).
semantic_highlighting1

  • 3. Understand code: The real facility that helps you understanding global code-structure is the navigation-tooltip or the code-browser. However those are not very useful to understand local algorithms. The following picture illustrates my favorite part of the semantic highlighting: Local Variable Colorization. That colorization assigns a semi-unique color to each variable in a local context. This allows much easier distinguishing those variables, largely without reading their full name at all. By freeing you of actually having to read all the variable names, this allows grokking local code relations faster, and has already helped me fixing quite a few very stupid bugs right away.🙂
    semantic_highlighting

    And the best thing about it: You don’t have to use it at all. Today I’ve added the option to completely disable semantic highlighting or local variable colorization all together. Although if you do it, be assured that you will re-enable it after a short time anyway.😉

    Development Update
    I’ve fixed tons of bugs, implemented a lot of new code-completion features, improved the internal template-support in general so it once gain works correctly with recent versions of the STL iterators, and most importantly, I’ve done a quite large change to the internal environment-management that makes parsing of large projects much more efficient, and scale a lot better disk-space wise.

    All together, I think I’ve pushed KDevelop4 far enough to be able to use it effectively on my upcoming Diploma Thesis, which gives me a very good feeling, and which represents a kind of milestone, since I won’t be able to put a similar amount of time into KDevelop4 in the next time as I did in the past.

    The C++ support now seems nearly feature complete, and very stable. I really haven’t encountered a non-temporary C++-support/duchain crash for a long time.

    Now we just need to push the debugger and the other lacking parts of KDevelop to come up to the expectations, and we’ll be heading for a very good release. As always, if you want to see this stuff in a stable release soon, consider helping, since some of those other parts really need some love.

  • 29 Comments »

    1. This is looking sweet… I might have to give KDevelop a go again sometime soon. I’ve been using gvim with make for a couple of years, looks like Linux IDEs have come a good distance in that time.

      Comment by Anthony — January 8, 2009 @ 3:48 am

    2. Very cool, I look fwd to KDevelop4 making its way into kubuntu (Tried a build from trunk and ran into heaps of problems).

      Comment by Lindsay — January 8, 2009 @ 5:41 am

    3. And if you’re color blind? ^_^

      Comment by pa — January 8, 2009 @ 6:07 am

    4. Or have synthaesthesia?

      Comment by Tom Mann — January 8, 2009 @ 7:58 am

    5. synaesthesia even..

      Comment by Tom Mann — January 8, 2009 @ 7:59 am

    6. @pa, Tom Mann:
      a) You don’t care, or b) you shut it off

      @Lindsay:
      Would have been interesting for us to know of the problems you ran into

      Comment by David Nolden — January 8, 2009 @ 8:54 am

    7. I agree with you that syntax higlighting really helps to understand and find errors.
      Actually if you’re looking for good usability ideas on syntax highlighting, you should really take a look at the latest eclipse (jdt).

      Since 2004 (eclipse 3.0), it has these features:
      ‘Dynamically marking occurrences in file’
      ‘Highlight method exit points’
      ‘Mark locations of thrown exceptions’

      see : http://archive.eclipse.org/eclipse/downloads/drops/R-3.0-200406251208/eclipse-news-part4-R3.html

      Comment by shamaz — January 8, 2009 @ 9:32 am

    8. @pa:
      Being color blind dies not really mean that one sees the world in black and white! From http://en.wikipedia.org/wiki/Color_blindness:

      “Color blindness almost never means complete monochromatism. In almost all cases, color blind people retain blue-yellow discrimination, and most color blind individuals are anomalous trichromats rather than complete dichromats. In practice this means that they often retain a limited discrimination along the red-green axis of color space although their ability to separate colors in this dimension is severely reduced.”

      Comment by Kishore — January 8, 2009 @ 9:41 am

    9. Kdevelop4 and DUChain are so awesome. 8> 8> 8> love love love❤❤❤

      The advanced code-completion makes learning unfamiliar libraries so much easier.
      🙂

      Comment by maninalift — January 8, 2009 @ 11:36 am

    10. The features DUChain gives us some great possibilities, making KDevelop useful for effective code editing, but when it comes to building the project, things seem to get worse. So the question is: is there any hope that building will become easier? What I mean is the ability to create a (default) project, e.g. Cmake (or Qmake) project, click “Build” and get it built? As far as I understand this now you have to go and configure a lot of settings by hand just to build it.

      Comment by thorGT — January 8, 2009 @ 1:00 pm

    11. @thorGT: That’s one of the parts of KDevelop that does need some more work. Although in the case of CMake the required setup is minimal. All you need to do is set up a build-directory in the project CMake settings, and you’re ready to go. That option is just a bit too deeply buried.

      Comment by David Nolden — January 8, 2009 @ 1:04 pm

    12. Just out of interest, how much of these features work for C as currently implemented? Obviously C would be an easier language to DUChain-ify as it’s (almost) a subset of C++. Will the existing features work out of the box on C sources? Or will you have to disable a lot of functionality?

      Comment by Parker — January 8, 2009 @ 1:35 pm

    13. Where are the colors defined? Are the customizeable? (I prefer the zenburn colors when coding, for example).

      Comment by sandsmark — January 8, 2009 @ 1:59 pm

    14. @Parker: I have tried it out a long time ago, and the C++ duchain does work quite well with C. All the features that are a subset of C++ aren’t a problem at all. The problem is the rarely used C-only things, like those strange argument-declarations behind the function-declaration. Those cause syntax errors on the C++ parser.

      Imo the C++ parser should be adapted to also work with the C specialties, because it’s not much missing. Nothing I’m planning to work on though.

      @sandsmark: Currently these colors are not customizable.

      Comment by David Nolden — January 8, 2009 @ 2:30 pm

    15. This is looking awesome, might get KDevelop4 soon (don’t think there are packages for Kubuntu yet), I don’t think I’ve ever see highlighting this advance before, and it certainly looks like it’d help to make understanding code easier and cut out mistakes.

      I’m wondering how it deals with some things, for example if you type a variable name it gets coloured based on whether it’s a local variable, a member variable of the class or a global variable or constant it appears.
      What happens if you mis-spell a variable name? It doesn’t get highlighted I assume, but I’m thinking it could get a red underline or something to show that it’s probably an error? same with function names and types could work.
      Would really help cut out errors, and since the mechanisms are already there it could just be a case of “else { //underline the word }” at the end of whatever highlights them.
      Of course I haven’t in any way seen how this works and am really just guessing.

      Comment by HappySmileMan — January 8, 2009 @ 4:49 pm

    16. Recently I’ve been working on log tracking, error reporting, and usage data analysis tools at work. We use queuing software to deliver messages around the globe, and to help correlate graphs and numbers to the names of the queue, I wrote some code that would hash the queue’s name into a number and use the number to influence RGB values. The hash was similar to the one Java uses on strings. Since the queue names are grouped by application (e.g., SAP.OUTBOUND.GERMANY, SAP.OUTBOUND.SINGAPORE, TRADECOLLABORATOR.FOO, TRADECOLLABORATOR.BAR, etc), the color hashing made it very simple to visually discern related queues on a large graph without requiring label clutter.

      I tend to write my variables the same way, such that related variables have similar names, the variables in your examples do too. So instead of random coloring, I’d suggest a name hashing coloring function. Could be useful!

      Happy hacking.

      Comment by Roy — January 9, 2009 @ 12:50 am

    17. @Roy:
      The colors are not random. A fixed count of colors is generated, which currently is 10, and then those colors are assigned to local variables. Hashing is used to pick the colors. This means that the same names will always get the same colors, except when the color is already taken by another variable.

      Comment by David Nolden — January 9, 2009 @ 1:12 am

    18. > What happens if you mis-spell a variable name? It doesn’t get highlighted I assume, but I’m thinking it could get a red underline or something to show that it’s probably an error? same with function names and types could work.

      From my past experience in Windows IDE’s: Not getting color highlighting is already enough to know something is wrong. You’ll seriously find yourself staring at the monitor like “hee.. why doesn’t it get coloured” and because the highlighting is always correct assume the bug is in your code.🙂

      Comment by Diederik van der Boor — January 9, 2009 @ 7:47 am

    19. Just a followup on the Eclipse comment. It goes even further — you can apply a simple rename “refactoring” on the currently selected identifier. Want to rename a local variable? Just put the cursor onto it, and invoke the inline rename (Ctrl-1 -> “Rename in file”) and the selected identifier, including all references to it, are renamed.

      Saves an awful lot of time.

      Anyway, great job you’re doing!

      Comment by gis — January 9, 2009 @ 10:23 pm

    20. […] C++ IDE Evolution: From Syntax Highlighting to Semantic Highlighting Most of us developers are so acccustomed to syntax highlighting, that we couldn’t live without it. Within the […] […]

      Pingback by Top Posts « WordPress.com — January 10, 2009 @ 12:03 am

    21. @gis: If you’d look back a few posts, then you’d know that KDevelop4 has that same feature for a few months already, and it doesn’t work reliably only locally, but globally.

      Anyway this post is on something completely different, and I’ve got enough of Eclipse/JDT feature enumeration.

      Comment by David Nolden — January 10, 2009 @ 4:06 am

    22. […] dies eine Chance um neue Funktionen und Ideen in KDevelop 4 einfließen zu lassen. Wie zum Beispiel Semantic Highlighting sowie automatische Include-Direktiven und […]

      Pingback by KDevelop 4 Beta 1 | OSZine — February 8, 2009 @ 8:25 pm

    23. […] Kdevelop4 supports semantic highlighting. […]

      Pingback by Room for Sk3pticism » QT Creator 1.0 on Debian 5.0 — March 10, 2009 @ 7:21 pm

    24. Hi,

      I wish KDevelop 4 will support syntax highlighting of extension-less C++ files such as the standard C++ library files eg vector, string which don’t have an extension etc.

      Also support for finding/regex in extension-less headers would be a great feature too.

      Comment by BigCat — May 7, 2009 @ 9:13 am

    25. Hello, I want to make style coding style checker for C++ and your highlighter seems to be a good base for it.

      can you briefly tell me what parts of your project are needed to create a standalone tool for code checking?

      Comment by vasaka — June 30, 2009 @ 8:34 am

    26. Very cool! I’ve also thought about color generation for variable names, or other methods that would allow identifying symbols without reading.

      I just found this article looking for what is the state of the art in semantic highlighting for C++ IDEs. As this blog post is already a few years old, do you know if there have there been additional developments in this area since 2009?

      Comment by laanwj — November 7, 2011 @ 10:56 am

    27. […] a read-only one, at least. Sometimes pictures explains better than words. This is an example of semantic highlighting when mouse hovering on a C++ […]

      Pingback by aelog | Integrating Woboq Code Browser — March 31, 2014 @ 4:30 pm

    28. Are there any C++ IDE with live preview for Linux?

      Simple answer: no (not that I am aware of)… However, there are many advantages to learning the standard Linux toolchains. And ways that you can get around some of the lack of “live preview” I’m not sure what advantage “Live preview” gives you wh…

      Trackback by Quora — August 5, 2015 @ 5:13 pm

    29. […] être employée pour montrer les flux de données dans la lignée de ce qui a été fait avec Semantic Highlighting (cache). Certains langages seraient certainement plus faciles à gérer que d’autres. Certaines […]

      Pingback by Code et couleurs Actualités — August 29, 2015 @ 4:14 am


    RSS feed for comments on this post. TrackBack URI

    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s

    Create a free website or blog at WordPress.com.

    %d bloggers like this: