Yesterday, during the talk of Ivan Zoratti, at the meeting with Marten Mickos and the Italian Team of SUN | MySQL in Rome, there was a question about Falcon performance: a guy pointed out that InnoDB is better than Falcon.

Well, this is not really the truth.

How many processors are you using for Falcon benchmarking?

Falcon is designed to make optimal use of modern large-memory multi-CPU/multi-core hardware. So when comparing performance of Falcon and InnoDB, you can’t leave this out of consideration!

Please, see the shoots below:

To learn more on the Falcon engine, you can attend this webinar:

• The Falcon Storage Engine in MySQL Server 6.0, by Jim Starkey, Principal Engineer/Senior Architect at MySQL AB.

Also note please that Falcon is in a beta stage, not alpha. The 6.0 binary may be labeled ‘alpha’ but it does contain the Falcon transaction engine beta!

More Resources on the Falcon Engine:

1. Falcon Feature Preview: http://forge.mysql.com/wiki/Falcon_Feature_Preview
2. MySQL Forum dedicated to Falcon: http://forums.mysql.com/list.php?133
3. MySQL 6.0 download: http://dev.mysql.com/downloads/mysql/6.0.html

Start playing with Falcon today! and thanks to the Falcon Team for their great and very interesting activities!

Yesterday, at the meeting with Marten Mickos and the Italian Team of SUN | MySQL in Rome, Giuseppe had a talk about the MySQL® Community.

If you are a newbie and you want to learn more about the fabulous MySQL® Community, please check this page the next week for his presentation!

Yo, don’t forget the MySQL® Magazine Survey:

1. proposal: http://www.paragon-cs.com/wordpress/2008/05/26/mysql-survey-online/
2. update: http://www.paragon-cs.com/wordpress/2008/05/28/mysql-survey-update/
3. take it now: http://www.kwiksurveys.com/online-survey.php?surveyID=KOKK_e414c40b

Great conference yesterday at “La Sapienza” University in Rome with Marten Mickos and the Italian Team of Sun | MySQL.

I’ve just returned in Triest, my hometown, after 8 hours :( train ride.

Slides should be available the next week, here.

Marten, thank you for coming, I hope to see you in Italy once again!

What would this world be without wascally wabbits, stuttering pigs, or suffering succotash? A century ago today, the voice of Bugs Bunny (and Daffy Duck, Porky Pig, Foghorn Leghorn, Pepe LePew, Wile E. Coyote…) let out its first utterance. Happy Birthday, Mel. Now check out what we drew up this week:

• Now, dammit: We’re all impatient, Blackberry-wielding, microwave-cooking, Tivo-recording fiends. Why should we wait a minute longer than we have to for our stock quotes? Yahoo! Finance gets that impulsive nature and found a way to get you real-time stock quotes, courtesy of BATS Trading. And it’s on the house. Free. Now. More here.
• Commons comments welcome: The Commons photo collection on Flickr just took the 2 train to Brooklyn. The Brooklyn Museum is the third institution (after the Library of Congress and Australia’s Powerhouse Museum) to add their hidden treasures to this public photography archive. You’ll find gorgeous photos from architectural research expeditions to Egypt and the Paris Exposition in 1900. But the best thing is that these images await your input and knowledge to make the collections even richer. So tag and comment away, for the good of humanity. More here.

Subscribe to the RSS feed (or add it to My Yahoo!) to get this Product Pulse every week.

Download Hyperic HQ

In This Issue

• Velocity Conference
• We’re Hiring!
• Vote for HQ in 2008
• Twittering Hyperic

• AOTMP Survey
• Upcoming HyperCASTs

• HQ 4.0 Iterations
• Best of the Blog

• News Roundup

New to Hyperic HQ?

Don’t forget to check out our community resources designed to ensure your success!

Upcoming Events

Velocity Conference

June 23-24

Burlingame, CA

Velocity Conference

O’Reilly’s Velocity Conference is only weeks away! This is a conference we helped design and build just for our users, the web operations teams powering some of the coolest companies on the planet. As such, Hyperic is a Diamond Sponsor and our own Javier Soltero will deliver a keynote “Clouds Are no Substitute for Competence.” With cloud computing lurking as the next big disrupter in the web operations world, you’ll want to hear what Javier has to say. And, for being a friend-of-Hyperic, you get a hefty discount!

To claim your 20% registration discount, use this code: vel08hyp

• Visit the Velocity Conference site

We’re Hiring!

Come join a stellar team of people to build, promote, sell, and service world-class Hyperic monitoring technologies. We have a wide range of positions available, and one is sure to catch your eye.

• See the list of open positions and apply today!

Vote for HQ in 2008

No, we’re not talking about Super Tuesday (there’s plenty enough candidates in that race already). We’re talking about the SourceForge.net 2008 Community Choice Awards! If you think Hyperic HQ deserves some recognition for the hard work and determination it has shown you this year, cast your vote for HQ in the category of your choice.

• Click here to vote for HQ

Twittering Hyperic

If you’re not familiar with Twitter, then where have you been for the last 2 months? When the occasional tweet isn’t getting you out of a foreign prison or helping you report on up-to-the-minute world events, it can also be a great way to follow something you care about. Like Hyperic! We’ll soon be updating the website and forums to take advantage of this new form of communication with our community of users and interested onlookers.

• Read the blog post here
• Click here to subscribe to our tweets

Call For Survey Participants

AOTMP is the industry authority on best practices and standards for managing enterprise voice, wireless, and data network services. AOTMP is seeking to better understand enterprises? challenges in network management for an upcoming report on industry trends and best practices, and who better to tell them than you, faithful reader?! Respondents to this survey will be enrolled in a drawing for opportunities to win a US$200 travelers check.

• Click here to take the survey

Upcoming and Archived HyperCASTs

Can’t get enough HQU? Want to get every last drop of performance out of MySQL? Would you like to see into Hyperic’s technological future? We’ve got you covered with these upcoming HyperCASTs. Click to RSVP.

• Practical Development of HQU Plugins & ResourcesJon Travis, Engineering and Marie McGarry, Documentation

  June 3rd, 2008 11:00 PDT / 18:00 UTC

• Using HQ to Performance Tune MySQLMike Mennis, Quality Assurance

  June 17th, 2008 11:00 PDT / 18:00 UTC

• Product Roadmap ReviewChip Witt, Product Management and Charles Lee, Engineering

  July 8th, 2008 11:00 PDT / 18:00 UTC

If you’ve never seen any of our HyperCASTs, we have built a library of archives. Some of the recent ones include:

• Customizing Hyperic Enterprise Reports
• Tuning HQ Health
• Building JMX Plugins
• Building SNMP Plugins

Hyperic HQ Iterations for 4.0

Our engineers have been taking a community approach in plotting the develpment of Hyperic HQ 4.0. One of the most visible changes wrought by this change in approach is the publishing of iterations, very early pre-alpha versions of code, on SourceForge.net. If you’re very curious and love to tinker and hack on developmental code, take a look:

• Hyperic HQ 4.0 Iterations
• Hyperic HQ 4.0 Roadmap - Learn about the progress and goals of the HQ 4.0 release

Best of the Blog

• Hyperic Announces MySQL Performance Study Results - A very cool study of the performance and scale we’ve been able to achieve using MySQL with HQ.
• VMotion Sickness Epidemic Worse Than We Thought - Stacey Schneider offers some shocking analysis of the effects of Virtualization on otherwise normal, healthy SysAdmins.
• Atlassian Using Hyperic - We noticed something cool recently on their wiki. As big fans and users of Atlassian’s work, it’s nice to see the feeling is mutual.

Hyperic in the News

• Using agent-based monitoring despite reservations - Megan Santosus writes about the Contegix case study, including mention of Contegix’s use of Hyperic HQ.
• Guide to Server Management - Denise Dubie writes about virtual server management, noting Hyperic HQ as an open source tool that can manage virtual servers.
• Red Hat Heads For The JON 2.0 - Sean Michael Kerner writes about the JBoss ON announcement. Hyperic is mentioned in a quote from Katrinka McCallum about our involvement.

In preparation for the book’s launch next month, I’ve created a website for it: High Performance MySQL. You may notice that the URL isn’t the same as the site for the first edition. It proved to be difficult to transfer that domain. If we accomplish it later on, I’ll set up a redirect.

Why an official site? To give you free stuff, of course. Final drafts of the front matter (TOC, preface, foreword), a sample chapter, and the index are there already. When the final quality control is done, I’ll update these. Right now they don’t have professionally drawn figures. That will change soon.

Also, you’ll eventually various things such as errata* and book-related info that I feel belongs there instead of here. You can subscribe to the site’s RSS feed to find out when these planned additions become reality.

* Surely there will be no errata, right? Right?

mysql

I've opened a 'feature request' bug on this at http://bugs.mysql.com/bug.php?id=37107

Many third party tools designed for MySQL are designed such that referential integrity constraints may only be defined on InnoDB tables. Up until recently, this was fine as a hard coded limitation because only InnoDB supported referential integrity. Unfortunately, the reality now is that many storage engines, especially pluggable ones have many varied capabilities. There is no SQL accessible method for determining exactly what specialized capabilities any given engine has. The INFORMATION_SCHEMA.ENGINES metadata table includes XA, TRANSACTIONS, and SAVEPOINTS columns, but these don't go nearly far enough.

We need a table that lets engines expose exactly what capabilities they feature.

TABLE ENGINE_CAPABILITIES
ENGINE VARCHAR(64)
CAPABILITY VARCHAR(64)
ENABLED BOOLEAN

MYISAM TRANSACTIONS FALSE
INNODB TRANSACTIONS TRUE
KDB TRANSACTIONS TRUE
FALCON TRANSACTIONS TRUE
INNODB XA TRUE
MYISAM XA FALSE
KDB XA FALSE
INNODB REFERENTIAL TRUE
KDB REFERENTIAL TRUE
MYISAM REFERENTIAL FALSE

As we’ve said before, no one ever wins the Yahoo! Lottery. And that’s simply because there is no Yahoo! Lottery. To protect you from these scammers, we’re going after them ourselves.

We recently filed a lawsuit that ought to send an unmistakable message to spammers masquerading as Yahoo! lottery commissioners, sending emails to unsuspecting users about having won a mythical jackpot. And in order to claim that jackpot, these “lucky” users simply need to hand over personal data like passwords, credit card information, and social security numbers. Some “winners” are even duped into sending money for processing and mailing charges.

We won’t tolerate these hoax emails or having our brand used to deceive you, and we’re seeking maximum damages permissible by law. This lawsuit (one of nearly ten suits to date) is part of a multi-faceted approach we’ve been taking to combat spam by supporting anti-spam legislation, creating technologies like DomainKeys, collaborating with industry leaders, and increasing consumer awareness. Consider this: every day, we block more than one billion spam and phishing messages — that’s more that four times the number of first class letters the US Postal Service delivers each day.

Besides what we’re doing on our end, here are some of the things you can do to protect yourself:

• Protect your email address like your phone number: Give it out selectively and only to people you trust. Don’t post it in public places like message boards or chat rooms. And try disposable email addresses (like our AddressGuard) for things like shopping or selling things online.
• Just say no to junk mail: Report unsolicited email by clicking on the spam button in the toolbar at the top of your inbox or message. This reports the contents so that Yahoo! Mail or your service provider can take appropriate action and potentially block them from reaching your inbox in the future.
• If it’s too good to be true, it probably is: Don’t be fooled by cash prizes or that prince who needs your help smuggling millions of dollars out of his country. They’re scams. Don’t reply, don’t click, and don’t give away any personal information.
• That’s no warning: See a pop-up ad with a warning about your computer? Or have an email from a “computer expert” warning you of a virus? They’re usually hoaxes from unscrupulous folks. Ignore them and don’t follow any steps described unless you’re sure the threat is real.
• Create a sign-in seal: Sign-in seals are a new safeguard offered by Yahoo! and many financial institutions to help protect your login. A sign-in seal is a secret message or image that you create so you can be sure you’re logging into your account and not a phishing site. To create your seal, go to any sign-in page across Yahoo! and look for the box with keys above your login.

You can find more tips on our anti-spam resource site. It’s a jungle out there, but there are plenty of ways to defend your inbox. Plus we’ve got your back.

Mark Risher
Anti-Spam Czar, Yahoo! Mail

Welcome the the 99th edition of Log Buffer, the weekly review of database blogs.

Since PGCon recently happened right here in Ottawa, let’s start with some posts about it, and about PostgreSQL. Josh Berkus came to the conference with his Database Soup. It sounds like he enjoyed himself: “So, that’s pgCon. It was exciting and fun. All of you PG geeks who missed it should be kicking yourselves about now, and putting in budget requests for next year.” He has day one highlights; day two highlights, and also some photos from the conference.

On Tending the Garden, Selena Deckelmann (who knows a thing or two about Postgres conferences) has her PGCon review, with links to conference webpages.

Robert Treat of zillablog also has his take-aways. “The biggest thing to come out of this past week to me was recognition of the continued growth of the postgresql community. With a number of regional conferences springing up over the past year, I wondered if PGCon would be able to match the experience from last time, and was happy to see that it far exceeded it. . . . One thing I noted was that we have a very large presence of Postgres in Ottawa, I think larger than what we had at PGEast for the Baltimore/Washington Area.”

On Greg’s Postgres stuff, Greg Sabino Mullane has an item on verifying master-slave replication with check_postgres.pl, a montoring script for Nagios and other systems.

Hubert Lubaciewski of select * from depesz; looks at different approaches to counting the number of distinct elements, such as distinct sessions per day.

On An Expert’s Guide to Oracle Technology, Lewis Cunningham examines the recent news of the worlds largest database running on Postgres. How large? 2 Petabytes! “Let’s put that in perspective,” writes Lewis. “1 petabyte is 1,000,000,000,000 bytes. Yahoo has two of those. . . . [They use] this database to analyze the browsing habits of its half a billion monthly visitors. How would you like to tune those queries?” Lewis also notes that this is not the plain-vanilla PostgreSQL that you or I would download and run on our website box.

There are conferences on the horizon of the Oracle world too. Dan Norris reminds us that ODTUG is just around the corner. That’s the Oracle Developer Tools User Group’s Kaleidoscope 2008 event, taking place in mid-June in New Orleans.

Gareth Roberts of In Depth Apps announces the call-for-papers of the New Zealand Oracle Users Group’s (NZOUG) 2008 conference, coming in October.

In other Oracle matters — Chen Shapira, just a simple DBA on a complex production system, has some thoughts on cumulative distribution. “How do we calculate Cumulative Normal Distribution? This means calculating the probability that a random variable from a normal distribution, will end up equal to or less than X.”

(more…)

I just saw that Atlassian, the provider of the essential community tools like Confluence wiki and JIRA ticket system, updated their wiki on the importance of monitoring the “lifeblood of your organization”.

They even outline the important monitoring tasks you need, and stress that it will help when dealing with their own world class support.

Monitoring involves a number of essential tasks, including those listed below:

• Monitoring log files.
• Checking for HTTP-availability and performance (e.g. by getting the same page every five minutes and displaying the time on a graph).
• Looking at many different parameters such as load, connections, IO, database-trends, and so on.
• Charting long-term trends.
• Keeping an access log of requests to the web server. This is vital, especially when requesting performance-related support from Atlassian.

They even pass on a screenshot of their Hyperic HQ deployment, which if you notice, they are hosted by Contegix, another Hyperic customer.

We agree 100% with Atlassian that it is critical to monitor their apps, which is why we’re also working with them to build application specific management plugins for Confluence and JIRA.

I will be offline for a few days.  This afternoon I leave Durham, North Carolina behind and going south once more to Pensacola, Florida — my hometown.  I am beginning a new job next week.  Since I haven’t asked if I can post about it I guess who my new employer is will remain anonymous for another few days.  Next Tuesday I will be flying up to the northeast corner of the U.S. to begin training.  Looking forward to it!

On another note..don’t forget the survey!  We are having great results but I would really like to ask everyone who uses MySQL in a professional role to please participate.  The larger the number of people who take ten minutes to fill out the survey the better the results.  Survey here: MySQL Survey

I would like to take a minute and publicly thank iContact (http://www.icontact.com) for the exciting work that I have done of late.  I had the honor to work with a talented group of people who are dedicated to their work.  Thanks all!!!

I'm pleased to announce the release of Memcached UDFs for MySQL. This release contains the ability to now pass expiration values. For instance, you can:

select memc_set('keyabc', 'somevalue')

or

select memc_set('keyabc', 'I die in 10 seconds', 10);

The functions that now have this ability are:

memc_set()
memc_set_by_key()
memc_append()
memc_append_by_key()
memc_prepend()
memc_prepend_by_key()
memc_add()
memc_add_by_key()
memc_replace()
memc_replace_by_key()

You can download the latest archive from http://patg.net/downloads/memcached_functions_mysql-0.5.tar.gz

Also, mercurial repository at http://hg.tangent.org/memcached_functions_mysql/

Please enjoy using these functions. I enjoy developing them! Thank you for using them.

Many vendors are going to performance based testing, also known as hands on exams. Most of the criticisms I receive on a regular basis that that all Certification Exams are PICKY or rely too much on the memorization of trivia. 'Besides, a real {DBA|Developer} never uses most of those commands in Real LifeTM!'

Well, your next MySQL Exam just may eschew most of those complaints. Developers could be asked to normalize tables, DBAs could be asked to set up replication, and Cluster DBAs could be asked to set up a full cluster. There are many details to be worked out.

One of these details are tools that do not come with your MySQL Server. I find the MySQL Administrator a very valuable tool for quick, day-to-day DBA tasks. I am probably not alone in using it. But would it be fair to allow it on a hands on exam? How about other tools?

So, if YOU have an opinion, please let me know and help guide the MySQL 6.0 Exam development.

Properties:

Applicable To MySQL Server Server Startup Option --log-queries-not-using-indexes Scope Global Dynamic Yes Possible Values Boolean Default False Category Performance, Monitoring, Best Practices

Description:

If you have slow query logs enabled (with --log-slow-queries), this variable will help interpret all those queries that are not using indexes as slow queries.

Usage:

While designing a MySQL oriented application, I generally design my schema first without introducing any index (only exception being the PRIMARY ones). Get the application ready. Next enable log_queries_not_using_indexes and start index usage testing. Analyze all queries, introduce indexes one by one and you are all set to go. This helps a lot in rewriting queries, figuring out the best possible index etc. Many times I have seen the need for swapping WHERE clause CONDITIONS for making it possible for the query to use an existing index.

For already deployed systems' performance glitches and production outages, this flag do serve as a starting point.

Though as always, usage of a variable is subject to circumstances. For my conditions and usage pattern, this variable comes very handy.

Read more:

• MySQL manual entry on log-queries-not-using-indexes
• Slow Query Logs

Hope you enjoyed reading this.

My last post, titled T-Dose CFP, got a comment from Bobby that people reading my feed from Planet MySQL couldn't possibly understand my post because of the lack of context.

So let me repost it..

Geekdinner is an unformal dinner where geeks meet , here in Belgium , but also at other places around the world, Every couple of months we meet , have dinner and chat about geeky stuff , such as tech conferences, mysql, drupal, jboss and other topics.

One of these topics was T-Dose , The Technical Dutch Open Source Event, which has their Call For Papers / Presentations available , so if you want to present there .. you have to tell them.

Bert Boerland gave a talk about Drupal there last year and Some Abstract Type, aka Geert Vanderkelen , MySQL/ Sun has also been spotted there before.

Would this be enough context or do you folks need more ?

Is my idea that people click on links to get more info , or even that people understand what a CFP is wrong ?

Last week in Prague, I met with Mark Shuttleworth of Canonical at the Ubuntu Developer Summit for version 8.10.

myself, Mark Shuttleworth, Arseniy Kuznetsov (Sun, NetBeans), Matt Zimmermann (Canonical CTO)

As you would expect, we discussed Canonical’s and MySQL’s offerings to the market. And we pondered on possibilities for joint customers to have the best of both worlds: Wouldn’t it be great if MySQL users on Ubuntu could get the “best bits” from MySQL through the software delivery mechanisms of Ubuntu?

Mark drew the above picture on a piece of paper (which I used Omni Graffle to capture in .png form), and we decided to explore how the “curved orange” part of the customer experience could be improved.

Kwik Surveys is the site that's hosting the 2008 MySQL Magazine Survey. The site will be going offline from Friday May 30th at 15:00 GMT through Monday June 1st, 09:00 GMT to upgrade to faster servers.

Hopefully this upgrade will go off without any problems, but just in case, I'll have a backup of all the results and forward them offsite to another location.

Have you surveyed?MySQL DBA & Programming Blog by Mark Schoonover

A big thank you to all those who attended the memcached webinar today on which I was a panelist. I was told that there were more than 560 registrants.

The feedback I received directly and indirectly shows that there is a lot of interest about memcached. In the future, I hope to work again with MySQL/Sun on more memcached related webinars.

If you attended the webinar and have some suggestions, comments or questions, please contact me at fmashraqi at yahoo dot com or post a comment on this blog.

Special thanks to Jimmy Guerrero, Monty Taylor, Rich Taylor, Edwin DeSouza and Alex Roedling for their hard work in arranging the webinar. Also thanks to Brian Aker, Matt Ingenthron and Trond Norbye for their assistance at various phases.

In case you missed the webinar:

• webinar recording: Designing and Implementing Scalable Applications with Memcached and MySQL ( you'll need Webex player to view the recording )
• memcached webinar slides

Gained some interesting insight earlier this week at the SUN/MySQL APAC eHorizons event. A key thread in my (business track) session was that a) databases are not interchangable, and b) you plan for growth. If you don't, you can't grow, and your business might end (or at least be in serious pain).

For instance, if you build an application but don't put application logic into place that deals with separate master/slave connections (even if they talk to the same server for starters!) is a bad idea. It's a pain to fix up later, and sometimes it's just impossible (better believe it, I have helped customers who struggle wish exactly this issue). Not all reads should go to slaves, sometimes you need absolute up-to date info from the master. So the logic is not quite as simple as "all selects to the slaves"....

Hardware selection is a result of all kinds of architectural choices, and again has to be geared for the future. Sometimes companies acquire hardware first, and then discuss application architecture. Sorry, wrong order. Or when encountering scaling issues, they order big chunky hardware first and then call me. Nice bigger master server(s) can make perfect sense, and good commodity hardware slave systems. That's just general logic, details really do differ per situation.

Now the insight. My client's client wanted the hardware selection based on energy footprint in the data center. The choice went to SUN T2 based systems, which have 8 cores, each with 8 execution paths. So they say "hi I have 64 cores" to the operating system.

From talking with Peter Zaitsev I understand that the CPUs themselves are not that fast (single thread), so the power comes from the numbers under multithreading. Good for MySQL, overall, when it deals with regular queries on many connections. Just need to mind things like ALTER TABLE and other operations which are essentially single threaded. Diddums for long-running queries (reporting).
For deployment, the most efficient solution will likely be having a few MySQL servers running on the same box, i.e. a master and a few slaves. That way some of the contention with single threaded operations is mitigated.
As with many of these things, "Try it And See" is the way, and so we will.

Anyway, the interesting take-away from this is that energy footprint is now also a factor when selecting hardware infrastructure. It makes sense, and in this case it should work out fine. There are multiple physical servers so there is redundany even with slaves sharing the same box with a master.