Just a reminder that the Toronto DemoCamp is tonight. Come out to see some cool Eclipse demos, talk to other Eclipse enthusiasts (and committers!), and have some free refreshments.
And yes, the weather is a bit crappy today, but we're still on! The snow is supposed to stop this afternoon.
Hope to see you at DemoCamp!
Thursday, November 22, 2007
Thursday, November 15, 2007
One Week Till Toronto Demo Camp
We have some cool demos lined up:
- Marcelo Paternostro, EMF: Building an Ecore model and seeing what EMF can do with it
- Mike Kucera, CDT Using Parser Generators To Build CDT Parsers
- Shaun Smith, Eclipse Persistence Services (EclipseLink) & Dali JPA Tools--Object/XML and Object/Relational mapping in Java
- Eugene Kuleshov, Task management and team collaboration with Eclipse Mylyn (Mylyn)
Thursday, October 18, 2007
Paintball, The "Other" Planet Eclipse, and A Humble Request
The other day a bunch of us here in the lab went out paintballing as a sort of team-building event. Little did the poor saps know that I used to play every single weekend, have all my own gear, etc. Needless to say the other team was frequently punished by the steady rain of purple paint from my tricked out Classic Automag .68 hammering down upon them. Muwahahah.
Oddly enough, going out with my co-workers has bitten me with the paintball bug again, so when waiting on a build I was checking around online to see how much some of the new markers I like cost. While looking around, I saw for sale a "Planet Eclipse 07 Ego" paintball marker. "There's an Eclipse paintball marker? COOL!!!" Intrigued, I Googled further, and it seems that http://www.planeteclipse.com is the website for the paintball company called (you guessed it) Eclipse that makes this particular marker.
They even have a marker that comes in a cool purplish-blue colour that sort of fits with the theme of our favourite IDE...
So instead of giving away Eclipse mountain bikes this year, how about some cool Eclipse paintball swag? This is way cooler than a mountain bike...
Here's hoping anyway :-)
Oddly enough, going out with my co-workers has bitten me with the paintball bug again, so when waiting on a build I was checking around online to see how much some of the new markers I like cost. While looking around, I saw for sale a "Planet Eclipse 07 Ego" paintball marker. "There's an Eclipse paintball marker? COOL!!!" Intrigued, I Googled further, and it seems that http://www.planeteclipse.com is the website for the paintball company called (you guessed it) Eclipse that makes this particular marker.
They even have a marker that comes in a cool purplish-blue colour that sort of fits with the theme of our favourite IDE...
So instead of giving away Eclipse mountain bikes this year, how about some cool Eclipse paintball swag? This is way cooler than a mountain bike...
Here's hoping anyway :-)
Friday, October 5, 2007
BeerClipse Meetings?
I just read Ian's blog about demo camps.
An interesting thought.
In addition to this sort of one-time thing, perhaps we could do something like the 2600 meetings. Set a standard time, like the second Friday of the month or something like that, to be the standard day for meetings. Then, people could locally organize their own meetings at that time at their local foodcourt/pub/whatever.
The 2600 meetings range from organized events with actual agendas and presentations, to informal "let's show up and shoot the breeze about hacking" meetings. The nature of the meeting is up to the organizers.
I think you could do something similar for Eclipse and have it be pretty successful. If nothing else. the promise that committers will be there will definitely draw in the user and contributer community because they'll want to corner us and ask questions. And if they don't show, there would always be stuff that we committer folk could talk about amongst ourselves over some frosty beverages.
What do people think about this idea? Would you participate on a regular basis?
An interesting thought.
In addition to this sort of one-time thing, perhaps we could do something like the 2600 meetings. Set a standard time, like the second Friday of the month or something like that, to be the standard day for meetings. Then, people could locally organize their own meetings at that time at their local foodcourt/pub/whatever.
The 2600 meetings range from organized events with actual agendas and presentations, to informal "let's show up and shoot the breeze about hacking" meetings. The nature of the meeting is up to the organizers.
I think you could do something similar for Eclipse and have it be pretty successful. If nothing else. the promise that committers will be there will definitely draw in the user and contributer community because they'll want to corner us and ask questions. And if they don't show, there would always be stuff that we committer folk could talk about amongst ourselves over some frosty beverages.
What do people think about this idea? Would you participate on a regular basis?
Tuesday, September 25, 2007
Java Makes It Hard to Find Good Help These Days...
From the Google-alerts-are-cool department:
Leif Frenzel has posted an interesting blog entry on the difficulties of creating tools for non-Java languages in Eclipse. He argues that in order to have tools that rival JDT, you have to have complicated tools that have intimate knowledge of the syntactic structure and semantics of the user's code, and this is hampered by the inability to code plug-ins easily in anything but Java. It's hard to find for example a parser generator that can handle Haskell, unless you're willing to use a parser generator written in Haskell itself. Go read it. Interesting food for thought.
Leif argues that this makes it hard for the community to contribute, because inherently the people that are most interested in seeing support for the language in question are people that are heavy users of that language, which also means that it's likely that their primary language of choice is not Java.
This makes sense to me because I've seen it first hand when trying to hire new members of my team that works on CDT. It's difficult to find people that have both good Java skills (needed for writing plug-ins), and a good knowledge of C and C++ (needed to know what the heck our plug-ins should be doing).
Would we find it easier to get help on CDT (both in terms of new hires, and in terms of help from the community) if people could write plug-ins in C++? Hmm... maybe...
Food for thought anyway. Personally I prefer writing my plug-ins in Java, thanks.
Leif Frenzel has posted an interesting blog entry on the difficulties of creating tools for non-Java languages in Eclipse. He argues that in order to have tools that rival JDT, you have to have complicated tools that have intimate knowledge of the syntactic structure and semantics of the user's code, and this is hampered by the inability to code plug-ins easily in anything but Java. It's hard to find for example a parser generator that can handle Haskell, unless you're willing to use a parser generator written in Haskell itself. Go read it. Interesting food for thought.
Leif argues that this makes it hard for the community to contribute, because inherently the people that are most interested in seeing support for the language in question are people that are heavy users of that language, which also means that it's likely that their primary language of choice is not Java.
This makes sense to me because I've seen it first hand when trying to hire new members of my team that works on CDT. It's difficult to find people that have both good Java skills (needed for writing plug-ins), and a good knowledge of C and C++ (needed to know what the heck our plug-ins should be doing).
Would we find it easier to get help on CDT (both in terms of new hires, and in terms of help from the community) if people could write plug-ins in C++? Hmm... maybe...
Food for thought anyway. Personally I prefer writing my plug-ins in Java, thanks.
Monday, September 10, 2007
Blog to Blog: Diversity
I was going to post a comment to Bjorn's blog entry about diversity, but I thought my response was food for thought enough (and controversial enough!) to merit its own blog entry. These thoughts have been simmering for a while.
I think diversity is good. A lot of projects need more diversity. However, I think that until the Platform has a non-IBM committer that is doing more than just contributing builds of SWT for a non-primary platform, then diversity at Eclipse is somewhat of a farce. The Platform, if anyone, should be a shining paragon of diversity and openness to the rest of the projects on Eclipse.org, but it's the number one project that people complain about with respect to diversity and openness.
It is very hard to get contributions in to the platform (often with good reason, we don't want crap in there obviously...), and as a result, it is hard to build up enough of a reputation to become a platform committer. While the platform team are good at posting plans and information, the processes that go into them are essentially closed. The day to day discussions about design and development issues do not seem to happen in an open forum. There are no public conference calls, and the mailing lists don't have significant traffic about the actual development going on in the platform. Instead they usually amount to status reports on builds and testing, peppered with the odd user question (which usually ought to have been posted to the newsgroups instead). And no folks, Bugzilla on its own does not count... not everyone has time to keep hitting refresh on Bugzilla to see what bugs are appearing and how they are going to be fixed. People at least need a heads up that "Hey, we're discussing $BIG_ISSUE over on Bugzilla #whatever, go there if you're interested in the discussion." Also, I would say that Bugzilla is not the greatest forum for discussing broader issues and project plans, etc.
In this sort of environment, it's hard to build a sense of camaraderie with the team. There is a definite "US and THEM" atmosphere. Not that the Platform people are adversarial, don't get me wrong. But, you definitely don't get the feeling they are actively trying to get newcomers into the fold. It's difficult to get enough information about what's going on in order to try to get up to speed to the point where you're on somewhat of an equal footing.
Now to be clear. I'm not trying to slag the Platform or the people that work on it. They have done a ton of awesome work over the years and we all owe a lot to them. Theirs is no easy job at the best of times. But, I do think there are things that they and other projects could be doing better, and so I'm trying to criticize constructively. Hopefully everyone takes it as such.
Now, let's contrast the status quo with the Platform project with my experience with becoming a committer on CDT. The company I was working for at the time was looking to move their IDE to the Eclipse/CDT platform. As I started coding up our integration, I started hanging on out the cdt-dev mailing list, reading and eventually participating in the in-depth requirements gathering and technical discussions that went on there. CDT conference calls were (and still are) open to the public, so I just started showing up. At first I didn't have a lot of useful things to say other than introducing myself and communicating that our company was starting to use CDT and was looking to contribute, but over time as I got ramped up, I had more meaningful things to say.
After fixing a few bugs on the 2.0 release, CDT 3.0 came along, and there were some features that I needed implemented that the commiters (of which I wasn't one yet) indicated that they didn't have time to work on. So what did I do? Well I hopped on the planning conference call and told them that I was committing to deliver patches for those features for the release. "Great!" they said, and put my items on the plan with my name next to them. I did what I promised and after the release went out I was rewarded with a nomination and subsequent election to commit rights.
The difference was the CDT gang went out of my way to make me feel like I was part of the team, even if strictly speaking I didn't have commit rights yet. Discussions and information were open enough that I could participate as nearly a first class citizen. Sure, I still had to submit patches for anything I wanted to change, and convince someone that those patches were worth committing, but that's much easier to do when you've already been collaborating as near-equals for a while. The open and collaborative nature of the project allowed me to build up an important thing: trust.
Anyway, back to the point.
I think really that the focus of the diversity rules and enforcement should not be to stop projects from starting that are not yet diverse, but instead should focus on opening up projects that already exist but are not diverse. I don't think diversity rules should get in the way of contributions. It's better to have a non-diverse project than no project at all, because it's better to have imperfect code that does something for someone, rather than have no code, which does nothing for everyone.
That being said, if a project exists and people can't contribute even when they want to, then that is a definite problem which we should be trying to fix.
Admittedly the devil is in the details, especially with a project as large and as widely consumed as the Platform, but without trying to sound heavy-handed, I think we should be looking at ways to open up it and other projects. Such changes would have to be practical. Maybe, for instance, you can't have a 100% open Platform Conference Call where anyone and everyone that dials in can say whatever they want, because you could get far too much signal to noise. But, maybe you can have a moderated call, where some people who are known and trusted (e.g. Eclipse committers) can speak freely, and others can flag questions for the moderators attention. This way information still flows, but hopefully supporting that flow is not onerous for the committers.
At any rate, I'm curious to see what's going to happen with this new diversity push. The Platform Debug team has been setting a good example as of late, and is actively trying to mentor in some new committers on the Debug sub-project. Kudos to them, and hopefully we will start seeing more things like this.
I think diversity is good. A lot of projects need more diversity. However, I think that until the Platform has a non-IBM committer that is doing more than just contributing builds of SWT for a non-primary platform, then diversity at Eclipse is somewhat of a farce. The Platform, if anyone, should be a shining paragon of diversity and openness to the rest of the projects on Eclipse.org, but it's the number one project that people complain about with respect to diversity and openness.
It is very hard to get contributions in to the platform (often with good reason, we don't want crap in there obviously...), and as a result, it is hard to build up enough of a reputation to become a platform committer. While the platform team are good at posting plans and information, the processes that go into them are essentially closed. The day to day discussions about design and development issues do not seem to happen in an open forum. There are no public conference calls, and the mailing lists don't have significant traffic about the actual development going on in the platform. Instead they usually amount to status reports on builds and testing, peppered with the odd user question (which usually ought to have been posted to the newsgroups instead). And no folks, Bugzilla on its own does not count... not everyone has time to keep hitting refresh on Bugzilla to see what bugs are appearing and how they are going to be fixed. People at least need a heads up that "Hey, we're discussing $BIG_ISSUE over on Bugzilla #whatever, go there if you're interested in the discussion." Also, I would say that Bugzilla is not the greatest forum for discussing broader issues and project plans, etc.
In this sort of environment, it's hard to build a sense of camaraderie with the team. There is a definite "US and THEM" atmosphere. Not that the Platform people are adversarial, don't get me wrong. But, you definitely don't get the feeling they are actively trying to get newcomers into the fold. It's difficult to get enough information about what's going on in order to try to get up to speed to the point where you're on somewhat of an equal footing.
Now to be clear. I'm not trying to slag the Platform or the people that work on it. They have done a ton of awesome work over the years and we all owe a lot to them. Theirs is no easy job at the best of times. But, I do think there are things that they and other projects could be doing better, and so I'm trying to criticize constructively. Hopefully everyone takes it as such.
Now, let's contrast the status quo with the Platform project with my experience with becoming a committer on CDT. The company I was working for at the time was looking to move their IDE to the Eclipse/CDT platform. As I started coding up our integration, I started hanging on out the cdt-dev mailing list, reading and eventually participating in the in-depth requirements gathering and technical discussions that went on there. CDT conference calls were (and still are) open to the public, so I just started showing up. At first I didn't have a lot of useful things to say other than introducing myself and communicating that our company was starting to use CDT and was looking to contribute, but over time as I got ramped up, I had more meaningful things to say.
After fixing a few bugs on the 2.0 release, CDT 3.0 came along, and there were some features that I needed implemented that the commiters (of which I wasn't one yet) indicated that they didn't have time to work on. So what did I do? Well I hopped on the planning conference call and told them that I was committing to deliver patches for those features for the release. "Great!" they said, and put my items on the plan with my name next to them. I did what I promised and after the release went out I was rewarded with a nomination and subsequent election to commit rights.
The difference was the CDT gang went out of my way to make me feel like I was part of the team, even if strictly speaking I didn't have commit rights yet. Discussions and information were open enough that I could participate as nearly a first class citizen. Sure, I still had to submit patches for anything I wanted to change, and convince someone that those patches were worth committing, but that's much easier to do when you've already been collaborating as near-equals for a while. The open and collaborative nature of the project allowed me to build up an important thing: trust.
Anyway, back to the point.
I think really that the focus of the diversity rules and enforcement should not be to stop projects from starting that are not yet diverse, but instead should focus on opening up projects that already exist but are not diverse. I don't think diversity rules should get in the way of contributions. It's better to have a non-diverse project than no project at all, because it's better to have imperfect code that does something for someone, rather than have no code, which does nothing for everyone.
That being said, if a project exists and people can't contribute even when they want to, then that is a definite problem which we should be trying to fix.
Admittedly the devil is in the details, especially with a project as large and as widely consumed as the Platform, but without trying to sound heavy-handed, I think we should be looking at ways to open up it and other projects. Such changes would have to be practical. Maybe, for instance, you can't have a 100% open Platform Conference Call where anyone and everyone that dials in can say whatever they want, because you could get far too much signal to noise. But, maybe you can have a moderated call, where some people who are known and trusted (e.g. Eclipse committers) can speak freely, and others can flag questions for the moderators attention. This way information still flows, but hopefully supporting that flow is not onerous for the committers.
At any rate, I'm curious to see what's going to happen with this new diversity push. The Platform Debug team has been setting a good example as of late, and is actively trying to mentor in some new committers on the Debug sub-project. Kudos to them, and hopefully we will start seeing more things like this.
Wednesday, August 15, 2007
Staffing the #eclipse IRC channel
For a long time our wonderful one of our beloved committer reps has been harassing us committers (and rightly so) in the hopes that more of us will hang out on the #eclipse channel on irc.freenode.net and help out with user questions about our various components.
This is arguably a noble goal, but in the past I had always felt that I just plain didn't have the time to devote to this. There are a lots of new user type questions that pop up on the newsgroups and mailing lists (to the point where a couple of years ago Webmaster had to tuck the mailing lists away in a corner on the website because the lists were becoming a free-for-all). This is especially true of the platform newsgroups and mailing lists. The fact that there are so many of these types of posts is a real testament to the popularity of Eclipse though, so it's not something to complain about either. But, in essence, I was afraid that there would be a bad signal to noise ratio for me, i.e. lots of basic questions about the platform and not a lot about CDT. Given that I have other things to do with the bulk of my time, I didn't figure I could keep up with the thread of conversation in a timely fashion, because it's not like I'd have the window up in front of me while I was working (sorry zx, I haven't tried ECF yet).
Lo and behold, I found something neat a few weeks ago which changed my mind. The Chatzilla plug-in for Firefox has this cool feature called "stalk words."
My previous experience with IRC was circa the late nineties, back when I was hacking away on and doing the writing for a video game total conversion that ended up, sadly, dying (but more relevantly, I was hanging out in IRC all day and night when I should have been doing my CS assignments). Back then IRC was not really state of the art, and if you wanted to do anything cool like this you had to write mIRC scripts yourself.
Chatzilla however, makes it pretty dirt simple. Basically, you can tell Chatzilla that there are certain phrases that you are particularly interested in, and when someone types one of them, Chatzilla can be set to alert you by playing a sound and/or flashing in the task bar. Particularly useful for us committers if you want to known when people are asking questions about the Eclipse project you work on.
This has been working out pretty well for me, and I've been fielding questions about CDT for a few weeks now. If you're a committer and your project is underrepresented in IRC right now, you might want to try it out too.
This is arguably a noble goal, but in the past I had always felt that I just plain didn't have the time to devote to this. There are a lots of new user type questions that pop up on the newsgroups and mailing lists (to the point where a couple of years ago Webmaster had to tuck the mailing lists away in a corner on the website because the lists were becoming a free-for-all). This is especially true of the platform newsgroups and mailing lists. The fact that there are so many of these types of posts is a real testament to the popularity of Eclipse though, so it's not something to complain about either. But, in essence, I was afraid that there would be a bad signal to noise ratio for me, i.e. lots of basic questions about the platform and not a lot about CDT. Given that I have other things to do with the bulk of my time, I didn't figure I could keep up with the thread of conversation in a timely fashion, because it's not like I'd have the window up in front of me while I was working (sorry zx, I haven't tried ECF yet).
Lo and behold, I found something neat a few weeks ago which changed my mind. The Chatzilla plug-in for Firefox has this cool feature called "stalk words."
My previous experience with IRC was circa the late nineties, back when I was hacking away on and doing the writing for a video game total conversion that ended up, sadly, dying (but more relevantly, I was hanging out in IRC all day and night when I should have been doing my CS assignments). Back then IRC was not really state of the art, and if you wanted to do anything cool like this you had to write mIRC scripts yourself.
Chatzilla however, makes it pretty dirt simple. Basically, you can tell Chatzilla that there are certain phrases that you are particularly interested in, and when someone types one of them, Chatzilla can be set to alert you by playing a sound and/or flashing in the task bar. Particularly useful for us committers if you want to known when people are asking questions about the Eclipse project you work on.
This has been working out pretty well for me, and I've been fielding questions about CDT for a few weeks now. If you're a committer and your project is underrepresented in IRC right now, you might want to try it out too.
Thursday, July 12, 2007
CDT Webinar
Just a reminder to everyone that Doug and I are presenting a webinar on CDT today. Here's the blurb that Lynn sent out recently:
==========================
CDT 4.0 – Reaching for Uberness
July 12, 2007 at 8:00 am PDT / 11:00 am EDT / 3:00 pm GMT/UTC
Presented by Doug Schaefer and Chris Recoskie
To register email webinar-cdt at eclipse dot org
C/C++ Development Tooling (CDT) 4.0 is our most exciting release yet bringing C/C++ developers a load of new features, quality, and performance improvements. This webinar presented by Doug Schaefer, the CDT Project Lead, will walk through all of CDT’s features from new project creation, code editing, and source navigation, to build and debug with a special focus on what’s new in CDT 4.0. New users will gain an understanding of how to use the CDT with their projects and all users will see how the improvements in CDT 4.0 make it a world class C/C++ development environment.
For more information on this and other Eclipse webinars visit http://live.eclipse.org/. Special thanks to Adobe for contributing their Adobe Acrobat Connect product to host the webinar.
==========================
Hope to see you all there!
Edit: In case you missed it live, here is the link to the recorded webinar: http://live.eclipse.org/node/293
==========================
CDT 4.0 – Reaching for Uberness
July 12, 2007 at 8:00 am PDT / 11:00 am EDT / 3:00 pm GMT/UTC
Presented by Doug Schaefer and Chris Recoskie
To register email webinar-cdt at eclipse dot org
C/C++ Development Tooling (CDT) 4.0 is our most exciting release yet bringing C/C++ developers a load of new features, quality, and performance improvements. This webinar presented by Doug Schaefer, the CDT Project Lead, will walk through all of CDT’s features from new project creation, code editing, and source navigation, to build and debug with a special focus on what’s new in CDT 4.0. New users will gain an understanding of how to use the CDT with their projects and all users will see how the improvements in CDT 4.0 make it a world class C/C++ development environment.
For more information on this and other Eclipse webinars visit http://live.eclipse.org/. Special thanks to Adobe for contributing their Adobe Acrobat Connect product to host the webinar.
==========================
Hope to see you all there!
Edit: In case you missed it live, here is the link to the recorded webinar: http://live.eclipse.org/node/293
Tuesday, June 12, 2007
Eclipse & CDT In The Scientific Community
After putting myself at the mercy of an extensive background check, I spent a week recently at Oak Ridge National Laboratory in Tennessee for a workshop centred around the Eclipse Parallel Tools Project and its usage for development of high performance, scientific applications. We had about thirty or so attendees from industry, academia, and the various U.S. national laboratories.
The workshop was largely strategic in nature (although Beth Tibbitts, Greg Watson, and Craig Rasmussen did give a packed tutorial on Eclipse, PTP, CDT, and Photran at the lab the day before the workshop). Basically we split up into groups and tried to come up with the challenges and deficiencies that we have right now with respect to Eclipse, PTP, performance analysis, debugging, and language support and services (such as the CDT).
That generated a giant list of action items, which everyone then got to vote on and hopefully sway the future development plans of all us tools developers in the room.
Greg has posted the meeting report, but in summary, two major themes won out.
1. We need remote tools.
- These developers are creating massively parallel programs on big iron. They don't have these machines sitting on their desks, they are typically in another building, or for some, in another city. They often can't run Eclipse directly on these machines due to lack of Eclipse support (and often the OS lacks any graphical UI whatsoever). Even if they can run Eclipse on it, for some the slowdowns due to all the network traffic required inhibit their productivity significantly. Generally since these huge machines are a scarce commodity their utilization rate is high and this can mean a lot of people using the machine at the same time.
- These applications are massively parallel for a reason. It's because they are doing huge simulations and/or crunching huge amounts of data. Performance is key because even in a best case scenario it often takes days to do a run. You don't want those days to turn into weeks, and so a large amount of development time goes into tweaking performance. Unfortunately, there isn't really much in the way of performance tools integrated into CDT or Photran right now. These people either need support for existing performance tools, or some new ones provided somewhere by Eclipse.org.
I also have lots of great pictures from the tour of the lab that they gave us that I'll be sharing soon. I just have to remember to upload them when I'm at home sometime.
Tuesday, May 15, 2007
The Europa Train - Blessing or Curse?
A bit of a controversial topic today...
I've always had mixed feelings about the simultaneous releases of Eclipse. It's definitely great to be able to go to one place and download a family pack of nuggets of Eclipse goodness (mmm.... nuggets...), and know that they all work together. Each year the release makes a big splash in the media too, which is great for Eclipse and everyone involved. That kind of publicity will likely make Eclipse.org the victim of its own self generated Denial Of Service attack as the download servers are brought to their knees by the throngs of downloads. That is definitely the kind of popularity and success that we all want.
One thing I've never liked about it though is all the peer pressure to make sure the train is on time. Don't get me wrong, I love having the train on time just as much as everyone else. It's nice to be able to know when the train is going to arrive so that you can plan your ride. Your boss is not going to be happy if you tell him you have absolutely no clue when you will be in to work because you don't know when the train is coming. But, if you knew that parts of the train were broken, wheels were missing, etc., wouldn't you want the mechanics to take the time to fix it, rather than it breaking down mid trip, forcing you to whip out your multi-tool and a pack of gum and MacGuyvering a solution? (BTW, I hereby declare MacGuyvering to be a word.)
My point is, the train sometime feels less like a train and more like a juggernaut. Last year for Callisto, we shipped a CDT 3.1.0 that had a lot of problems with it. CDT 3.1 was a big release that had a lot of new functionality, and it's inevitable that when you deliver a lot of new content there are going to be lots of problems, through absolutely no fault of the committers and contributors. Even good code inevitably has a certain amount of bug, and so it follows to at least a certain extent that the more new code you put in, the more bugs there will be. As some for-instances, there were a lot of problems with the indexer, and search was pretty much broken at the time. Yet, we shipped anyway, because the train had to be on time. It's often said to ISVs that consume CDT that they shouldn't take the dot-zero because it will be buggy, instead take the dot-one. I hate to say it, but really IMHO CDT 3.1.0 should have been baked for a while longer.
I don't think this is the greatest situation. Individual projects ought to be able to hold up the train if they need to. However, along the lines of a recent post by Doug Schaefer, I think Eclipse gets itself sometimes into a situation where it's a victim of its own hype. The date for the release is picked a year in advance. We spend so much time in that year hyping the release that by the time we start getting into the bug fixing cycle, there is already so much pressure to release on time that we couldn't hold things up if we wanted to. We shouldn't be releasing things that we know for a fact that people shouldn't be using. Speaking in practical terms, a dot-zero release is never going to be flawless, but if you're shipping with major functionality broken, or with crippling bugs that preclude widespread adoption, then I think the purpose for the release has been somehow lost sight of. A release that can't really be used is somewhat pointless.
Now, I'm sure someone is going to reply to this and say something along the lines of "the release is at the whims of the committers", and that we really have the power to hold things up. Technically it may be true by the letter of the process, but if you really believe that I suggest you try it and see what happens. Short of the Platform or JDT being horribly busted, I'm pretty sure you will get voted down.
I think that what needs to happen is that the release needs to be bug count driven. This process is not flawless either, but the idea is that you do what we do on the CDT milestones, and we don't do the build until all the bugs targeted for that milestone are fixed. When we reach Zarro Boogs, then it's Go Time. Sure, nefarious people can play games still by spuriously marking bugs RESOLVED - INVALID, or by playing games with severeties and target milestones or what have you, but I think that on the whole the idea works. This way, the date of the release is driven from the bottom up by the committers, and is not imposed on them from above for them to deal with after the fact. Sure, you still need to give a rough estimate to people as to when they can expect something (e.g. "Summer 2008"), and the committers shouldn't be given license to delay as long as they please without reason, but at least then there is some flexibility built into the plan.
Don't misread what I am saying here either. CDT 4.0, which is coming out on the Europa train, is shaping up to be both the most feature rich and yet most robust version of CDT yet. I'm not currently anticipating a recurrance of what happened on 3.1.0, and I would definitely recommend to users of previous versions of CDT that they move up to 4.0 and take advantage of the scores of bug fixes and new features it includes. And, I also think having a release train is on the whole a good thing. But, I think there are some things we can do so make sure everyone gets a say in how the train operates.
I'm curious to see how next year's release will unfold...
I've always had mixed feelings about the simultaneous releases of Eclipse. It's definitely great to be able to go to one place and download a family pack of nuggets of Eclipse goodness (mmm.... nuggets...), and know that they all work together. Each year the release makes a big splash in the media too, which is great for Eclipse and everyone involved. That kind of publicity will likely make Eclipse.org the victim of its own self generated Denial Of Service attack as the download servers are brought to their knees by the throngs of downloads. That is definitely the kind of popularity and success that we all want.
One thing I've never liked about it though is all the peer pressure to make sure the train is on time. Don't get me wrong, I love having the train on time just as much as everyone else. It's nice to be able to know when the train is going to arrive so that you can plan your ride. Your boss is not going to be happy if you tell him you have absolutely no clue when you will be in to work because you don't know when the train is coming. But, if you knew that parts of the train were broken, wheels were missing, etc., wouldn't you want the mechanics to take the time to fix it, rather than it breaking down mid trip, forcing you to whip out your multi-tool and a pack of gum and MacGuyvering a solution? (BTW, I hereby declare MacGuyvering to be a word.)
My point is, the train sometime feels less like a train and more like a juggernaut. Last year for Callisto, we shipped a CDT 3.1.0 that had a lot of problems with it. CDT 3.1 was a big release that had a lot of new functionality, and it's inevitable that when you deliver a lot of new content there are going to be lots of problems, through absolutely no fault of the committers and contributors. Even good code inevitably has a certain amount of bug, and so it follows to at least a certain extent that the more new code you put in, the more bugs there will be. As some for-instances, there were a lot of problems with the indexer, and search was pretty much broken at the time. Yet, we shipped anyway, because the train had to be on time. It's often said to ISVs that consume CDT that they shouldn't take the dot-zero because it will be buggy, instead take the dot-one. I hate to say it, but really IMHO CDT 3.1.0 should have been baked for a while longer.
I don't think this is the greatest situation. Individual projects ought to be able to hold up the train if they need to. However, along the lines of a recent post by Doug Schaefer, I think Eclipse gets itself sometimes into a situation where it's a victim of its own hype. The date for the release is picked a year in advance. We spend so much time in that year hyping the release that by the time we start getting into the bug fixing cycle, there is already so much pressure to release on time that we couldn't hold things up if we wanted to. We shouldn't be releasing things that we know for a fact that people shouldn't be using. Speaking in practical terms, a dot-zero release is never going to be flawless, but if you're shipping with major functionality broken, or with crippling bugs that preclude widespread adoption, then I think the purpose for the release has been somehow lost sight of. A release that can't really be used is somewhat pointless.
Now, I'm sure someone is going to reply to this and say something along the lines of "the release is at the whims of the committers", and that we really have the power to hold things up. Technically it may be true by the letter of the process, but if you really believe that I suggest you try it and see what happens. Short of the Platform or JDT being horribly busted, I'm pretty sure you will get voted down.
I think that what needs to happen is that the release needs to be bug count driven. This process is not flawless either, but the idea is that you do what we do on the CDT milestones, and we don't do the build until all the bugs targeted for that milestone are fixed. When we reach Zarro Boogs, then it's Go Time. Sure, nefarious people can play games still by spuriously marking bugs RESOLVED - INVALID, or by playing games with severeties and target milestones or what have you, but I think that on the whole the idea works. This way, the date of the release is driven from the bottom up by the committers, and is not imposed on them from above for them to deal with after the fact. Sure, you still need to give a rough estimate to people as to when they can expect something (e.g. "Summer 2008"), and the committers shouldn't be given license to delay as long as they please without reason, but at least then there is some flexibility built into the plan.
Don't misread what I am saying here either. CDT 4.0, which is coming out on the Europa train, is shaping up to be both the most feature rich and yet most robust version of CDT yet. I'm not currently anticipating a recurrance of what happened on 3.1.0, and I would definitely recommend to users of previous versions of CDT that they move up to 4.0 and take advantage of the scores of bug fixes and new features it includes. And, I also think having a release train is on the whole a good thing. But, I think there are some things we can do so make sure everyone gets a say in how the train operates.
I'm curious to see how next year's release will unfold...
Monday, May 7, 2007
Language Extensibility In CDT 4.0
This screenshot is really cool. But why?
We'll get back to the screenshot I promise.
Well, CDT 4.0 RC0 just went out the door last week, marking our first feature complete build for Europa. Confusingly enough our next build this week is going to be marked as M7, so we have the odd situation where we have a milestone build after our first RC, but the team felt it was important to keep the naming convention constistent with the Europa build of which we will be a part of, and we didn't want users getting confused about which build of CDT to use with Europa M7.
There are a few cool new features that my team here at IBM have been working on that a number of ISVs and other language tools authors are going to hopefully find useful. It's always been fairly easy to add support to CDT for compiling different languages via CDT's Managed Build and Standard Make projects, but we've been working recently to make it easier to integrate new C-like languages into the Core so all those cool features like search, open declaration, and content assist all work.
For a while now, it's been possible to contribute definitions for new languages into the CDT core. Circa CDT 3.1, we added an extension point to CDT to allow you to contribute new languages via the ILanguage interface, and to map those ILanguages to an Eclipse content type. Each ILanguage has methods it must provide that let you parse a file and get an Abstract Syntax Tree (AST) out of it as result. Once you have an AST all those cool features I mentioned eariler start working, provided you use CDT's DOM AST APIs.
This worked great for clients such as the Photran project (who do the Fortran language IDE integration for Eclipse), but it was a bit problematic if you actually wanted to override what the language was for C and C++ files. CDT would look for extensions to the extension point, but would stop looking once it found the first one for any given content type (I'm simplifying things but this is how it would appear from the user's point of view). Hence, there was no deterministic way to make sure that your language was the one that got used.
In CDT 4.0 we've now added the concept of language mappings to the workbench preferences and the project properties. What this means is that the user can go in and change which language is mapped to which content type, even down to the level of the individual file.
The language mapping feature is great for those that compile the same project on multiple platforms with different compilers that all support slightly different variations of C or C++. Now if you have a build configuration for each platform you can set the language mappings on each configuration individually, and your code will be parsed properly in each configuration (provided that you have an ILanguage to handle those scenarios). It's also great for embedded vendors, as most of them tend to have slightly different variations on the C programming language to enable you to do some cool things like handle interrupts, etc. This way they can define their own ILanguage which can handle these differences.
Another big thing we've been working on is making it easier to create the parsers for those language variants. The most frequently encountered use case for this stuff are the use cases belonging to people like those embedded vendors I mentioned. For the most part the language they are implementing is nearly identical to C or C++, and they just need to add a couple of keywords or a few new constructs. Up until now they've pretty much had to write a whole new parser for that from the ground up. The GNU C and C++ parsers that are bundled with CDT are lean, mean, hardcoded state machines, and they are pretty difficult to get your head around if you are brave enough to crack open the code; difficult enough that most people that want to integrate a new language variant into CDT pretty much gave up right there. Don't get me wrong, those parsers are great at what they do (and without them we'd have been parserless for years), but they were designed with peformance in mind, and not readability or maintainability. If you tried to extend from the concrete classes in order to modify the behaviour of the parser you'd end up overridding big gnarly functions that do most of the work, and so if we ever fixed a bug in the original parser it probably wouldn't trickle down to your code unless you looked for it and cut & paste it into the parser you created.
Enter the new parsers that my team has been working on. One of our core requirements from our internal customers here at IBM was support for new language variants. Since we knew we were going to have several variants to support over the next few years, it seemed like a worthwhile investment to create some kind of extensible parser framework. To keep things "simple" we started with C. The goal was to create a basic C parser based on the ISO C99 standard, and to make it reuseable to support other language variants. In theory then language implementers would get C parsing for "free" and could concentrate on just defining the delta of their language compared to the base language.
It seemed natural for us to to use a parser generator for this. Parser generators take as input a grammar which specifies the rules of a language, and from that grammar it generates a parser that can handle that language. Just having a grammar will let you recognize whether a set of input abides by the rules of the language, but generally you want to do more than that. Typically as well you would define semantic actions in your grammar that do interesting things, which in our case was build up an AST with CDT's DOM AST APIs, so that once the language was parsed all those cool tools I mentioned earlier could recognize the structure of the code and do Cool Things(TM) with it.
So, what we did was create a C99 parser using the LPG parser generator, which has semantic actions in it to build up an AST for CDT. LPG is a parser generator built by some folks at IBM Research, which is being used for the parser in Eclipse's JDT, as well as for the SAFARI IDE Generator. The cool thing about LPG is that it has a notion of language inheritance. What this means is that if you take our C99 grammar file, you can do the equivalent of a #include in your own grammar for your own language to pull in our grammar. You can then add new rules or overrride our rules as you see fit, i.e. you get C parsing "for free".
The results of this were pretty amazing. One of our requirements which we got from the Eclipse Parallel Tools Platform people was to support a new programming language coming out, Unified Parallel C, which is a variant of C for massively parallel applications. The language adds new keywords and constructs which allow you to control the parallelization and synchronization of your program. By including the C99 grammar in our UPC grammar, we were able to get UPC working in a matter of days. Time to go back to our screenshot of the CDT editor, with a UPC file open:
There's a whole lot of cool stuff going on there:
After CDT 4.0 is out the door we're going to start looking at doing some more interesting things with parsers.
We'll get back to the screenshot I promise.Well, CDT 4.0 RC0 just went out the door last week, marking our first feature complete build for Europa. Confusingly enough our next build this week is going to be marked as M7, so we have the odd situation where we have a milestone build after our first RC, but the team felt it was important to keep the naming convention constistent with the Europa build of which we will be a part of, and we didn't want users getting confused about which build of CDT to use with Europa M7.
There are a few cool new features that my team here at IBM have been working on that a number of ISVs and other language tools authors are going to hopefully find useful. It's always been fairly easy to add support to CDT for compiling different languages via CDT's Managed Build and Standard Make projects, but we've been working recently to make it easier to integrate new C-like languages into the Core so all those cool features like search, open declaration, and content assist all work.
For a while now, it's been possible to contribute definitions for new languages into the CDT core. Circa CDT 3.1, we added an extension point to CDT to allow you to contribute new languages via the ILanguage interface, and to map those ILanguages to an Eclipse content type. Each ILanguage has methods it must provide that let you parse a file and get an Abstract Syntax Tree (AST) out of it as result. Once you have an AST all those cool features I mentioned eariler start working, provided you use CDT's DOM AST APIs.
This worked great for clients such as the Photran project (who do the Fortran language IDE integration for Eclipse), but it was a bit problematic if you actually wanted to override what the language was for C and C++ files. CDT would look for extensions to the extension point, but would stop looking once it found the first one for any given content type (I'm simplifying things but this is how it would appear from the user's point of view). Hence, there was no deterministic way to make sure that your language was the one that got used.
In CDT 4.0 we've now added the concept of language mappings to the workbench preferences and the project properties. What this means is that the user can go in and change which language is mapped to which content type, even down to the level of the individual file.
The language mapping feature is great for those that compile the same project on multiple platforms with different compilers that all support slightly different variations of C or C++. Now if you have a build configuration for each platform you can set the language mappings on each configuration individually, and your code will be parsed properly in each configuration (provided that you have an ILanguage to handle those scenarios). It's also great for embedded vendors, as most of them tend to have slightly different variations on the C programming language to enable you to do some cool things like handle interrupts, etc. This way they can define their own ILanguage which can handle these differences.
Another big thing we've been working on is making it easier to create the parsers for those language variants. The most frequently encountered use case for this stuff are the use cases belonging to people like those embedded vendors I mentioned. For the most part the language they are implementing is nearly identical to C or C++, and they just need to add a couple of keywords or a few new constructs. Up until now they've pretty much had to write a whole new parser for that from the ground up. The GNU C and C++ parsers that are bundled with CDT are lean, mean, hardcoded state machines, and they are pretty difficult to get your head around if you are brave enough to crack open the code; difficult enough that most people that want to integrate a new language variant into CDT pretty much gave up right there. Don't get me wrong, those parsers are great at what they do (and without them we'd have been parserless for years), but they were designed with peformance in mind, and not readability or maintainability. If you tried to extend from the concrete classes in order to modify the behaviour of the parser you'd end up overridding big gnarly functions that do most of the work, and so if we ever fixed a bug in the original parser it probably wouldn't trickle down to your code unless you looked for it and cut & paste it into the parser you created.
Enter the new parsers that my team has been working on. One of our core requirements from our internal customers here at IBM was support for new language variants. Since we knew we were going to have several variants to support over the next few years, it seemed like a worthwhile investment to create some kind of extensible parser framework. To keep things "simple" we started with C. The goal was to create a basic C parser based on the ISO C99 standard, and to make it reuseable to support other language variants. In theory then language implementers would get C parsing for "free" and could concentrate on just defining the delta of their language compared to the base language.
It seemed natural for us to to use a parser generator for this. Parser generators take as input a grammar which specifies the rules of a language, and from that grammar it generates a parser that can handle that language. Just having a grammar will let you recognize whether a set of input abides by the rules of the language, but generally you want to do more than that. Typically as well you would define semantic actions in your grammar that do interesting things, which in our case was build up an AST with CDT's DOM AST APIs, so that once the language was parsed all those cool tools I mentioned earlier could recognize the structure of the code and do Cool Things(TM) with it.
So, what we did was create a C99 parser using the LPG parser generator, which has semantic actions in it to build up an AST for CDT. LPG is a parser generator built by some folks at IBM Research, which is being used for the parser in Eclipse's JDT, as well as for the SAFARI IDE Generator. The cool thing about LPG is that it has a notion of language inheritance. What this means is that if you take our C99 grammar file, you can do the equivalent of a #include in your own grammar for your own language to pull in our grammar. You can then add new rules or overrride our rules as you see fit, i.e. you get C parsing "for free".
The results of this were pretty amazing. One of our requirements which we got from the Eclipse Parallel Tools Platform people was to support a new programming language coming out, Unified Parallel C, which is a variant of C for massively parallel applications. The language adds new keywords and constructs which allow you to control the parallelization and synchronization of your program. By including the C99 grammar in our UPC grammar, we were able to get UPC working in a matter of days. Time to go back to our screenshot of the CDT editor, with a UPC file open:
There's a whole lot of cool stuff going on there:- Syntax highlighting of new keywords (upc_forall)
- Outline view works
- Content assist is finding constructs in the code
- Content assist is working on constructs that are not normally legal C!!! It's a subtle point, but take a look at where the caret is in the upc_forall statement. This construct takes four expressions, not the usual three that your plain old ordinary for loop takes. Yet, content assist in that fourth expression just plain works!
After CDT 4.0 is out the door we're going to start looking at doing some more interesting things with parsers.
- Firstly, we want to write a GNU C language variant on top of our C99 parser and see how that stacks up against the existing GNU parser in CDT in terms of correctness and performance. We're already re-using all of the parser JUnits on our parser, so I already have a warm fuzzy about correctness. If the speed is good enough then I would love to replace the old parser with one based on ours because then it will be a lot easier to maintain.
- Secondly, we're going to start tackling C++. Parsing C++ properly is a very difficult problem, given all the ambiguities in the language itself. I know personally of teams of people using bottom-up parsing techniques to parse C++ so I know it can be done (LPG is bottom-up too), but we have to figure out how feasible this is to do with LPG. Luckily we have a good line of communication with the LPG authors, and they are keen to see LPG being used successfully on C++, so if we encounter any roadblocks hopefully we can work together to smash them down.
Monday, April 23, 2007
Ride For Sight 2007
I am proud to support Ride for Sight and The Foundation Fighting Blindness' plight to discover treatments and cures for blindness. Help join the fight against blindness today! My personal goal is to raise $500.
You can help support me by making a secure online donation using your credit card. Click on the link below:
https://secure.e2rm.com/registrant/StartUp.aspx?SID=1444606
For more information on how YOU can participate in Ride for Sight, please visit us at http://www.rideforsight.com .
Thanks for your support!
Chris
You can help support me by making a secure online donation using your credit card. Click on the link below:
https://secure.e2rm.com/registrant/Start
For more information on how YOU can participate in Ride for Sight, please visit us at http://www.rideforsight.com .
Thanks for your support!
Chris
Subscribe to:
Posts (Atom)
