Tech Talk 5: Developing an Open Source IMAP Server
Join Mailtrust on Wednesday August 6th for a presentation by Timo Sirainen, the lead developer and creator of Dovecot, an open source IMAP and POP3 server.
When: Wednesday, August 6, 2008, 6:00 PM Eastern
Where: Mailtrust, 775 University City Blvd, Blacksburg VA
RSVP: techtalk@mailtrust.com (free pizza!!)
From the Virginia Tech campus, take the Tom’s Creek B bus to the first University City Blvd stop. The bus runs every 10 minutes.
Summary of the talk:
Dovecot is an open source IMAP and POP3 server for Linux/UNIX-like systems, written with security primarily in mind. Dovecot is used by both small and large organizations, including large email hosting companies such as Mailtrust. It’s fast, simple to set up, and included as the default IMAP server in popular Linux distros such as Red Hat. Timo will talk about the architecture of Dovecot, as well as what it’s like to run a popular open source project.
If you’re interested in the technical details behind an IMAP server, interested in creating your own open source project, or just generally curious about how the wacky world of open source software operates, please join us on August 6 to learn about Timo’s experiences.
Tech Talk 4: Agile Software Development
Join Mailtrust on Wednesday July 9th for a presentation by Richard Hammer of Modea covering Agile Development.
When: Wednesday, July 9, 2008, 6:00 PM Eastern
Where: Mailtrust, 775 University City Blvd, Blacksburg VA
RSVP: techtalk@mailtrust.com (free pizza!!)
From the Virginia Tech campus, take the Tom’s Creek B bus to the first University City Blvd stop. The bus runs every 10 minutes.
Summary of the talk:
“Agile” has quickly become a technology term of art and a center space in buzzword bingo. It encompasses several iterative and incremental software development methodologies, including: XP, Scrum, Lean Development, and Feature-Driven Development. While each of these methods is unique in its specific approach to solving today’s software development headaches, they all share a common vision and core values: The Agile Manifesto.
Fundamentally, all of the agile development methodologies incorporate iterations and continuous feedback to refine and deliver functional software systems. Key touch points in the processes include continuous planning, continuous testing, continuous integration, and other forms of continuous evolution of the project, the software, and communications between the teams. They are all lightweight and inherently adaptable, focus on empowering people to collaborate and make decisions together quickly and effectively. Basically delivering what is needed, when it is needed, while maintaining the ability to change course and adjust priorities with less risk.
Come see what the art form and buzz are all about. Share your experiences. Listen to the experiences of peers. A video of the talk will be posted shortly after it is given.
Hitting the the Road to Talk About the Cloud
We have a bunch of folks hitting the road over the next few weeks to get out and talk about Rackspace, Racklabs, Mosso and our other cloud initiatives.
Please look us up if you are attending any of these events.
One event is already history. John Engates spoke on a panel at Enterprise 2.0 about Cloud Computing. I am hoping they will post the video of this panel.
This Tuesday Jonathan Bryce and Todd Morey of Mosso will be at Stanford for the VentureLab event on Cloud Computing. They will discuss our approach to cloud computing and how we have incubated Mosso outside the mainstream of Rackspace.
The week of the 23rd is a big one in the Bay area for infrastructure and cloud events. Here is where you can find us:
- Jonathan will be at O’Reilly’s Velocity.
- He will do a short talk that night at Ignite. The rule is 20 slides in 5 minutes. I think he is up for it.
- Then its off to Cloudcamp. Not sure of the format yet, but we will be there to talk about our Cloud initiatives.
- Then Om Malik’s Structure08 conference. I think Om has really done a great job on this one. Great program, one action packed day. Many, many people in our industry are going. I will be on the cloud computing panel.
Hope to see you at some of these events. We will report back to you.
Rackspace Developers Conference 2008
Our first developer conference was held over the last couple of days and came to a close Friday afternoon. I’ll go ahead and declare it a success based on all the feedback I received from attendees. RackLabs and Mailtrust sponsored the event, but we had software developers from every corner of the company in attendance. In fact, we invited everyone at Rackspace who actively writes code as a part of their job and even some of the Rackers who just work closely with those developers.
We began planning this event early this year when a few of us were talking about how many developers we must have at Rackspace by now and how few of them likely know what one another is working on. We’re a bit spread out now with office locations in Texas, Virginia, and London and it makes it hard to keep up with what other teams are working on. So we decided to fix that…
Our dev conference was designed to bring all the Rack developers we could together in one place to expose them to some new technologies, to share ideas, and to meet a few new people or at least put names with faces. We decided to hold it in San Antonio because we could get the highest attendance here, so we flew in a bunch of the guys at Mailtrust down from Blacksburg. We thought about holding the conference at Rackspace HQ, but to cut down on distractions we needed to get offsite. We ended up renting out the old Southern Pacific Railroad Depot in downtown San Antonio. Nowadays it’s called Sunset Station, but it’s a historic building built back in 1902 and has a great atmosphere. We had lunch both days nearby in an old church.. What a cool venue. Lanham Napier, our CEO, came down and talked to the group on the first day and pointed out something pretty remarkable…back in 2000 we had the Rackspace holiday party at Sunset Station and today there are more developers at Rackspace than there were total employees back then. Crazy.
Our conference revolved around a series of technical talks delivered by Racker developers about various technologies used at Rackspace, Mosso, or Mailtrust. We even talked about a few technologies that we don’t use…yet! Here are a few of topics just to give you a little flavor:
- HA and Scalability @ Mailtrust
- MapReduce
- Botnets Gone Wild, Real Botnets, Real Wild
- Trends in Concurrent Programming
- Intro to PostgreSQL
- Asynchronous, event-driven programming with Twisted Python
- and many more…
We also had several “demo” booths set up to allow Rackers to demonstrate the technology they were working on and give others a chance to see it up close. One of the demo booths showed off our new cloud storage platform called CloudFS. Also demo’d was some of the cool technology we’ve built for integrating and scaling the immense monitoring infrastructure at Rackspace. Very cool stuff indeed.
It was exciting to see so many Rackspace developers in one place at one time and to hear about all the cool stuff they’re working on and thinking about. We’re planning on doing it again and may even invite some customers next time around. Stay tuned.
If you’re interested in joining Rackspace and working on some very challenging projects, we’re looking for good Python and Java developers. It’s getting interesting around here, so now is the time to join!
Here are some photos from our conference:
Our T-shirts. The front says /[RH]ACKER/
The place where we held the conference.
My “keynote” slides.
Bill Boebel, CTO @ Mailtrust
The old church where we had lunch.
A shot of the conference.
Some of the RackLabs guys at lunch. Eric, Jay, and Jay.
How to Build a Storage System
Challenge to Rack Labs from the business based on real customer need:
Build us a storage system that is the following:
-Quick to Market: We needed to develop the system with tools/ technologies that were well known at Rackspace and Mosso.
-Scalable: It needs to provide the same level of performance to 1,000 users as it does 100,000 users. The system must scale horizontally. As our customer base and their needs grow, so should our system.
-Dynamic: The space should grow/shrink with demand. Users shouldn’t need to do capacity planning and over-buy/under-utilize space, or worry about running out of space. Customers should only pay for what they use.
-Low cost: It needs to be inexpensive. We were targeting $0.15/ Gig.
-Developer friendly: Make it easily accessible to developers. It should support generic web-friendly interfaces as well as more traditional, language-specific technologies.
-Reliable performance: It should perform the same whether a customer is storing 5Gigs of data or 5TB of data.
-Redundant: Make sure that data will always be available. The system should keep multiple copies of the data and be built in a manner that is redundant at both the hardware and network layers.
-Secure: Data should be protected. Ensure that data is secure and only available to the specific customer. All traffic should be encrypted over SSL and data needs to be stored in an internal private network largely isolated from other Rackspace networks.
Rack Labs interpretation of this request:
How to build a Storage System - by Rack Labs
Business Design, the one liner:
Build us a Storage System that is easy to use, highly available, very secure, fast, extremely scalable, simple to maintain, cost effective for the customer and oh, it has to be done as inexpensively and as quickly as possible. A developer’s dream.
Lots of problems. Let’s knock ‘em out.
1. Problem: Finding good coders is hard.
Solution
Well first you find the smartest developer in the company and put him in a room and tell him you want a storage system. Give him the room he needs to experiment, create and exercise his full capabilities. When he starts showing some results, you add the second smartest guy in the company to the team and tell him to help. Dim the lights, turn on the AC and supply ample amounts of Ruta Maya coffee and more smart folks will join.
2. Problem: Gotta be cheap. No fancy gear, heck, all you get is what we got.
Solution
Rackspace is a hosting company. We have a ton of perfectly good older servers around that are not in use anymore. First of all you take some of those “seasoned” servers and you rebuild them from the ground up with exactly what you need. Heck we’ve got the parts. We don’t need lots of processing power. We don’t need much RAM. We need disk space and a lot of it.
Can we buy some big hard drives? Sure.
3. Problem: Build a super redundant system using a bunch of old servers.
Solution
Use a lot of them and make multiple copies of everyone’s data and stick them in different areas of the DC for good measure. Build the system so that if any one server dies, just plug another one in. The system is smart enough to start making copies when it sees a failure anywhere.
4. Problem: This system is going to run on a bunch of servers. That means provisioning and maintaining them. People are expensive.
Solution
We built a provisioning system that is completely plug and play. The servers are easy to build and we do that hundreds of times a day for our other customers. CloudFS servers can be slotted anywhere in the DC and they boot off the network. If one fails, we just stick another one in its place. Fast and easy. We also built a monitoring and control console that is heavily automated and the entire system can be by just a small crew of folks.
5. Problem: Build a publically accessible system that is privately secure.
Solution
This was a challenge but we figured it out. First of all we isolated the network so no one else in our DCs has access to it. Then we layered on all the fun encryption you find in SSL. We are confident that no one will know what anyone else is storing unless, of course, they want them to.
6. Problem: This system has to be easy to use. What is easy to us is not necessarily easy to everyone else.
Solution
Mosso and Rackspace. The hard-core developers can code directly to language specific APIs but Mosso and Rackspace will be adding interfaces that will make it simple enough for our parents to possibly use (still trying to keep email a mystery to them though!) Interested in being a partner in this? The door is open.
7.Problem: Quick to market. How many times have we heard this before?
Solution
This one was not easy to solve. We think “quick” is 2 years. They think it is 1. We hired a few more folks to help out and we stay up late at night. The nice thing is that we get to release something solid. Does it have all the bells and whistles we want? Not yet, but we are working on it. What the Beta testers will see is only the beginning.
The Result - Today Mosso is announcing a private beta for a new storage system called CloudFS. CloudFS is ready for testing. We are looking for a select group of beta testers to help us put it through its paces. We know we’ve got some more work to do to officially release this as a product, but there’s nothing like real users to help “push” us along. If you’re interested head to our CloudFS page on Mosso.
Want to help out? Know Python or Java? We need you. Email us: 0XC0FFEE@racklabs.com
(Posted on behalf of the Rack Labs team)
Tech Talk 3: Practical Web Semantics
Posted by Bill Boebel, Mailtrust
Join Mailtrust again on Monday April 28th for a presentation by Manu Sporny of Digital Bazaar covering the Semantic Web.
When: Monday, April 28, 2008, 6:00 PM Eastern
Where: Mailtrust, 775 University City Blvd, Blacksburg VA
RSVP: techtalk@mailtrust.com (free pizza!!)
From the Virginia Tech campus, take the Tom’s Creek B bus to the first University City Blvd stop. The bus runs every 10 minutes.
Summary of the talk:
A practical introduction to The Semantic Web and the technologies that enable web developers and bloggers to embed meaning, such as marking up people, places, events, music and locations, into websites. Areas covered will include Resource Description Framework (RDF) basics including CURIEs and N3 notation, implementation approaches such as Microformats and RDFa, and authoring tools such as Operator and Fuzzbot. The talk will be given by Manu Sporny, who is an Invited Expert to the World Wide Web Consortium (W3C), one of the primary RDFa Task Force members working on the RDFa specification and the primary author of the hAudio Microformat specification.
If you can’t make it, don’t worry… stay tuned to this blog for a video of the talk shortly after the event.