mysqlguy.net

Planet MySQL

Syndicate content
Planet MySQL - http://www.planetmysql.org/
URL: http://www.planetmysql.org/
Updated: 1 hour 12 sec ago

New in MySQL 5.1: The Presentation

2 hours 50 min ago

What's new, in a nutshell: http://dev.mysql.com/doc/refman/5.1/en/mysql-nutshell.html.

Release notes: http://dev.mysql.com/doc/refman/5.1/en/news-5-1-x.html (In the video, it's the page entitled "Changes in release 5.1.x").

And yes, very early on (at about 2 minutes in) I talk about my take on Monty's controversial post at http://monty-says.blogspot.com/2008/11/oops-we-did-it-again-mysql-51-released.html

read more

Categories: MySQL

You said you are a SALES engineer? What the heck is that?

December 26, 2008 - 8:32am
I'm a Sales Engineer, I admit it. This blog is mostly about technical stuff, but I felt I should write a not-so-technical post for once. For a techie like myself, and to someone else out there, even the Sales Engineer title may look weird. Frankly Sun doesn't have Sales Engineers, they have System Engineers, which is the same thing. Not. Not at all, it is something different. But I insist on being a Sales Engineer.

I have been an SE (as in Sales Engineer) on and off (mostly on) for a good 20+ years now, more or less always in the database industry (a short term with security software is the exception). I learnt the SE trade at another RDBMS company in the 1980's. I had joined them as Support Engineer, having previously been a developer. I learnt SQL the hard way, by using it. I learnt to use indexes by having not used indexes, realising things were slow, looked at the syntax and found the word INDEX which sounded like something that would speed things up, and it did.

Anyway, I now was working for a US corporation as a Support Engineer in Sweden. There was 8 of us. No, not 8 support engineers, 8 employees in Sweden. Only one Sales Engineer, and that was NOT me. I was not too keen on having to meet a customer, as I had been forced to be a trainer early on, and I was scared stiff by that experience. And the Sales Guys then. I hadn't dealt much with them previously, but now.. Oh my. So this Sales Guy comes into my room one day, the SE was off to something else, and a techie was needed at a customer meeting the next day. I was the only one available, so "Take your best suit, a white shirt and a tie and you are all set, it's easy", the Sales Guys said, trying to calm me down. It didn't. I didn't have a nice suit, nor a white shirt and a tie, well I did not ever want to wear one, and I didn't have one either.

But what could I do, I was the youngest employee by far, the lowest on the ladder and self-confidence wasn't something that was I too familiar with. So I got myself a suit, a shirt and a tie (in those days, if you were in Sales at all, you just HAD to wear a tie) and went along with the fast-talking salesguy. And I found out I liked it. So after some time, I went along, took some training and became an SE and was assigned the important task of carrying the Sales Guys cellphone (this is true. And cellphones were BIG in those days. And only management and salesguys had them).

Now, 20+ years later, what am I doing you ask? A consultant that isn't necessarily paid? A tech support service for sales guys and customers that Sales Guys think deserves it? Someone with the most contradicory title on the planet? Frankly, the job of an SE includes all those things. But the deal with an SE doing, say consulting or tech support isn't what we do, but why. We do engineering work in the name of Sales, and that isn't necessarily a bad thing. That's the what and the why.

As for any customers reading this, and a few of you have probably meet me out there, what does this mean? Do we fake engineering abilities so we can get paid? Nope, a Sales Engineer is always an Engineer at heart. If my customers are happy, then I am happy. We are honest guys, it's just what we do is for a different reason than, say, tech support. But that doesn't mean we are evil, at least I try to avoid it (Santa was here fo me this year, so I think I am doing OK).

Doing what is right, from a technical perspective, is what we try to achieve. For example, a prospective customer wants to start a project based on this or that idea, and he is using MySQL for it. It is not the Sales Persons job to say yes or no to that, or even to understand if this is a good thing or not. No, the sales persons job is to make sure that the customer gets a fair deal on MySQL for this project. But what if the design of the thing is completely flawed? Anyone can see that this will not work, right? Well, the customer might not have worked with MySQL ´before, and maybe not even with an RDBMS. And it's not the salespersons job to understand or question this (which doesn't necessarily mean that he doesn't understand, it's just that it isn't his job). So whose job is it then to get back to the customer with Sales person and say that, "Hey, this is not really such a good idea" or something to that effect? The SE. That is what we do. And also the SE should provide some alternative design here.

And if the customer says, "I want MySQL, but I'm not sure about the design", well an SE can answer that. So we can spend a few weeks at a site for free fixing the customers design, before any contract of any kind has been signed. Nope, wrong. There is a balance here, SEs don't do things "for free", as it seems sometimes. But we do help out, point out a direction and give advice, in the extent we can do just that.

What kind of training does an SE have, what competences to we hold? Typically more a combination of many things than in most other professions. An SE typically knows little about many things, and combine that with knowing A LOT of a few things. I'm a good C hack for example (hey, I am *old* now) and I know a few other things well too.

Is what I am saying here trade secrets of MySQL / Sun Microsystems then? Nope, if you have worked as a Sales Engineer, you know it's about the same thing everywhere. Sometimes we are called Pre-Sale consultants, sometimes SE, sometimes Field Engineers or something, but the job and what we do is similar.

We are all in all the engineering side of the Sales Process. When a sales person doesn't know if technical project has any chance of being successful, he asks us. When a prospective customer has some technical questions, we answer them or get them answered.

And then there is one other thing. When the customer has paid up, are we gone into thin air, never to be seen again. Nope, as I said, we are after all Engineers, we like to see technology work, we want to help, and as MySQL is selling subscriptions, we will see you again next year.

SE's with MySQL / Sun Microsystems has a lot of freedom, more than with any company I have worked for an an SE. If a technology or a customer interests me, I am allowed to dig into that and get my hands dirty, should I wish so. Aand are we cometing with the MySQL / Sun Microsystem Preofessions Services consultants? Nope, we try hard not to.

All in all, we are techies, the work we do is stuff that a Support Engineer or PS Consultant can do, at times we do those jobs at least (we are probably not as fast as those guys at it though). We just do these things for a slightly different reason. And we ARE a bit salesy, I admit, we have to know what is going on in the market, we have to understand how to explain something in a few words to someone who might not be a techie, but someone who is signing the checks, for a customer.

After so many years as an SE, more than most I think, I can say that being an SE at MySQL / Sun is great. And I look forward to more of it. And (blushing a bit) I a´was here easrly enough to start it (we were 2 SE's back then). I can tell you more about that, but this posting is long enough already. Next posting will be back at technology, I promise!

Yours truly
/Karlsson
Categories: MySQL

WFTO - Weekly Falcon Test Overview 2008-12-26

December 26, 2008 - 2:48am
WFTO - end of 2008 edition This is the last WFTO report for 2008. Our MySQL/Falcon team is mostly on vacation and therefore there was not that much activity last week. However, Sergey Vojtovich fixed one tablespace related bug. I am looking forward to report about our WFTO many more times in 2009! Since my last report [...]
Categories: MySQL

Amarok 2.0 uses MySQL

December 26, 2008 - 12:10am

I’ve always been more of a GNOME guy, and when running Linux, I use Rhythmbox to play my music. However, Amarok 2.0 might just change that.

They’ve chosen their database - it is none other than MySQL. The release notes state:

Some features, such as the player window or support for databases other than MySQL, have been removed because either they posed insurmountable programming problems, or they didn’t fit our design decisions about how to distinguish Amarok in a saturated market of music players.

If you want to know why the decision was made, read MySQL in Amarok 2 - The Reality. It has a lot to do with the fact that MySQL can be embedded, and performs well. Its a generally useful read to see why SQLite and PostgreSQL was not chosen.

MySQL… powering the music of today!
(wonder as I may, if we will ever get any Enterprise customers, who make heavy use of Amarok over many computers, etc… - I’m thinking modern night clubs, lounges, et al)

Categories: MySQL

OurDelta

December 25, 2008 - 5:35am
The MySQL packages in Debian still do not (and will never) include inofficial feature patches (beside the redirect output of error.log to syslog in MySQL 5.0, which is now part of the official MySQL 5.1 releases) like the ones from Percona or Google. Of course we add important bugfixes from newer (not yet released) versions of MySQL, but no feature patches.

If you are interested in a patched version of the Debian MySQL packages, take a look at OurDelta, their builds include patches (complete list) like mirrored binlog and InnoDB freeze. Upgrading from the official Debian packages of MySQL to the builds from OurDelta should work without problems, but please use Launchpad when reporting bugs against their packages, and not the Debian Bug Tracking System.
Categories: MySQL

MySQL 5.0.75 uploaded to Debian

December 24, 2008 - 6:45pm
This week MySQL released version 5.0.75 of their Community Edition. The package in testing is in good shape, so I had time to prepare and upload 5.0.75-1 to unstable, but it should work on testing as well. I also made a rebuild for etch, which is available here.
Categories: MySQL

Dutch and French winning, Italian close behind

December 24, 2008 - 9:01am

First of all, I'd like to thank everyone who has helped translating messages for Drizzle so far.

Secondly, congratulations to Dutch and French speakers, since you have translations of all translatable strings! (so far, Mark Atwood is working on an overhaul of the error message system, so I imagine we'll find some more strings that need work)

Italian speakers are close - there are only 76 of 1413 messages untranslated.

After the top three, things fall off a bit. German has 744 untranslated, English (UK) has 905 untranslated (although honestly, there probably aren't that many needing translation). Spanish has 1019 untranslated, Norwegian Bokmal has 1049 and Brazillian Portuguese has 1078. Hindi and Polish round out the top ten with 1243 and 1251 untranslated respectively.

So good job everybody, and thanks for all the hard work! For anyone who wants to pitch in who hasn't, you can just head over to http://translations.launchpad.net/drizzle and jump in

Categories: MySQL

Database Connection Node Changes

December 24, 2008 - 7:10am

Hi all,

As I mentioned in a previous blog entry, big changes are underway to the database connection connection node in NetBeans 7.0. If you look at the image above, you'll see what I mean. In the connection that is highlighted, no schema was selected when the connection was made, thus "default" is displayed as the schema.

When the schema is default, all schemas are displayed under the connection node when it is expanded. When you expand the schema, you'll see a familiar sight: the Tables, Views, and Procedures folders.

Individual database connections are also displayed as separate nodes below, even with default schemas.

Be sure to stay tuned for changes.

Happy Holidays!

--James

Categories: MySQL

Using triggers for performance

December 24, 2008 - 6:39am
Can you use TRIGGERs to increase performance? Really? Isn't so that a TRIGGER on a table will reduce performance, just as any FOREIGN KEYs will? Right?

Nope, none of those statements is necessarily very true. In the case of reads, for example, neither TRIGGERs, nor FOREIGN KEYs has any negative impact at all. Then, assuming we have a read-intensive system, such as some website, then performance isn't much of an argument for not using triggers or foreign keys.

You already know I'm quite a fan of foreign keys. Let me reiterate why I like them: They keep my data integrity intact, that's why. The argument that if I have transactions, then the only thing that might cause data integrity to fail is if there is a bug in the application. There are two things I have to say against that:

  • Many, if not most, users use more than one application. For example, if nothing else, the mysql and mysqldump commandline tools are used. To me, if I use foreign keys properly, I can safely mock around manually in the database with SQL, without causing data integrity issues.
  • Foreign keys are declarative. This means that the rather strict things they enforce are easy to look up and maintain, without looking into numerous pieces of code spread all over the place.


Now, I have made my point with Foreign keys, so what about the original subject of tjis article, i.e. Triggers to increase performance, all I have said so far is that they do not at least decrease performance for reads, but can they increase performance? Yepp. Let me give you an example.

Even with semi-complex data structures, you sometimes get into a situation where even a moderately normalized RDBMS structure cause some queries to be unnecessarily complex. On the other hand, denormalizing it usually has other issues. For example a de-normalized design is often best (performance wise) for SELECTs, whereas a normalized design keeps my data integrity better maintained, in particular with INSERT, UPDATE and DELETE statements. And this anomality is where triggers help!

This is how I think: Start with a pretty normalized structure. Identify the areas where a SELECT would be unnecessarily complex, then add the "de-normalizing" columns to the tables, without removing the normalized column structure or foreugn keys. This means I will have duplicate data, but it also means that SELECTs will be performaing much better. INSERTs, DELETEs and UPDATEs will be more complex and less performant, but this is no big issue as:

  • INSERT, UPDATE and DELETE are less frequent! I said this was a mostly read-only system, right!
  • And here is the magic sauce: I can use triggers to maintain the "denormalized" columns. My INSERTs, DELETEs and UPDATEs will not change at all compared to the normalized structure, as this is still in place as it always has been.


I had this issue the other day. I run a website on my spare time where I, among other things, keep track of bluesbands and gigs with them. I figured out that the bands usually were a combination of artists, i.e. I had "Fat Foo" on one gig, and on another I had "Fat Foo and the Bar". So I ended with having the latter being a lineup and the former an artist. In an ideal world, this would have been a great candidate for a hierarcical CONNECT BY, scenario, but not so now, without that particular feature in MySQL.

All the same, I realized, for a number of reasons, that this would be a bit sluggish. Every time I looked up a gig (in the gig table), got get the name of the artist I needed at least 3 lookups: The lineup, the lineup->artist table and the artist table. A gig always references just a lineup, so every artist is part of at least one lineup, the one with just one member:
Artist "Fat Foo" is the only artist in the Lineup "Fat Foo"
but
Artist "Fat Foo" is also one the members in the Lineup "Fat Foo and the Bar", the other artist in that lineup being "The Bar".

A lot of table lookups, just to get the name of a band. So I created not a trigger here (I'll soon tell you why), but a stored function that would update the corresponding lineups everytime an artist changed. The function would simply sync the lineup, having been given an artist ID.

Now it was to to create the trigger that would call the function above. So why didn't I embed the code in the function right into the trigger? Because I also wanted to make sure that I could always redo all the lineup names, if I decided I wanted to change the scheme I use for this. If I did that, I would use the function easily for that, with a simple SELECT.

So, now you ask, how does it work in practice? And what about the case when "Fat Foo" and "The Bar" keeps their names, but the combo is renamed to "The Fat Bar". I just have two columns in the lineup table, one which contains a "generated name" by the function above, and one which is manually entered. I use the manually entered name if it exists, else I use the generated one, a simple IFNULL will do for this, no extra lookups or anything!

Look at the result on PapaBlues.

And by the way, folks, Merry Christmas to you all
/Karlsson
Categories: MySQL

Does MySQL really have an open-source business model?

December 23, 2008 - 11:50pm
I’ve been thinking about the business of what’s variously come to be called commercial open-source and enterprise open-source. I’m interested in the gestalt — the product, development processes, marketing, licensing and so on. MySQL has tried many different ways to earn money. These include dual licensing, support subscriptions, a knowledgebase, consulting, an Enterprise/Community split, [...]
Categories: MySQL

MySQL Workbench 5.1 1st Alpha for Mac OS Available

December 23, 2008 - 8:57pm

Our Mac Developers have done an outstanding job in catching up with the Mac frontend. We have worked hard in the last days to get a little Christmas present out for all people waiting for a Mac Version of our Database Tool.
It’s a 1st alpha version, so don’t expect a full blown design application already - and the binary is Intel only this time (hey, it’s only Christmas folks ;).  But what we deliver is a working UI where you can load/save existing Workbench-documents and create diagrams from imported SQL-scripts. The canvas and the GRT system is up and running and there’s a shell which enables messing with the objects on GRT level. There are no object editors yet (so you can’t create and edit objects, but still build your diagrams for your existing SQL CREATE scripts) and wizards for exporting scripts, and connecting to databases are also missing. But enough writing, grab the .dmg package from our server and give it a try to get yourself a picture of what Workbench is going to look & feel natively on your Mac.  Note that Workbench for Mac requires Mac OS X Leopard.

Binary Package

ftp://ftp.mysql.com/pub/mysql/download/gui-tools/mysql-workbench-5.1.6-osx10.5-i686.dmg
ftp://ftp.mysql.com/pub/mysql/download/gui-tools/mysql-workbench-5.1.6-osx10.5-i686.dmg.md5

Source Package

ftp://ftp.mysql.com/pub/mysql/download/gui-tools/mysql-workbench-oss-5.1.6.tar.gz
ftp://ftp.mysql.com/pub/mysql/download/gui-tools/mysql-workbench-oss-5.1.6.tar.gz.md5

As example to get you started I’ll give a short step-by-step guide on how to import your script and assemble a nice diagram from it:

  1. (more…)

Categories: MySQL

5.0.75-build11 percona binaries

December 23, 2008 - 7:28pm

As you may know Sun / MySQL made release of community 5.0.75 only as source code release. We made binaries with our patchset.
Patches are mostly equal to build 10
http://www.mysqlperformanceblog.com/2008/12/11/mysql-binaries-percona-build10/, only change:
it includes bugfix for https://bugs.launchpad.net/percona-patches/+bug/308849

You can download binaries (RPMS x86_64) and sources with patches here
http://www.percona.com/mysql/5.0.75-b11/

Entry posted by Vadim | No comment

Add to: | | | |

Categories: MySQL

Merry Christmas!

December 23, 2008 - 7:15pm

I spent the last month taking a break from Open Source, partly due to FreeBSD 7.1 failing to hit the FTP servers as scheduled (any excuse), but mostly due to Santa’s early delivery of Ableton Live. Not really Open Source I may add but extremely cool nonetheless and a much needed break from coding!.

I find that staring too much at code can hamper the creative process (hehe), and as you know Creativity and Innovation go hand in hand with Open Source Software!.

Speaking of which, while I have been busy playing with waveforms and elastic audio, the Open Source world has been all guns blazing with cool releases like MySQL 5.1 GA, Open Solaris 2008.11, VirtualBox 2.1 and Wordpress 2.7 to name a few.
You really cannot take a break these days without something big in the world of Open Source happening in your absence.

So the year is nearly at an end which got me thinking, what was the highlight of 2008 in the world of Open Source?.
I have to confess to spending more time than I would have liked this year working with proprietary software, always in combination with Open Source tools of course!. So if I had to highlight anything from 2008 it would be that Open Source is now more pervasive than ever. From your OSX laptop at home to the Linux servers at work, right the way up to the Internet’s biggest sites and services (running FreeBSD ), you’ll find Open Source Software behind every successful venture. In fact it is now so popular even Microsoft could not resist the urge to get in on the game!.

In fact there are so many Companies using Open Source now that proprietary software may actually be the minority! We all know everyone loves an underdog, so maybe propreitary software will be the hit in 2009? Then again with the state of the economy maybe not!

The biggest influence for me this year (both good and bad) had to be Sun Microsystems who played a large part in Open Source adoption in the enterprise and continue to lead the way with their open business model (GlassFish, MySQL, Netbeans, OpenJDK, OpenSolaris and VirtualBox to name just a few of their many contributions).

Although most of the time it seems like they cannot do anything right, one thing is for sure, without the corporate weight of Sun behind the Open Source ethos we may not have made it here so quickly. So while your saving big bucks with your ZFS storage and processing millions of transactions with MySQL, don’t forget to say thank you to Sun next time you are complaining about JavaFX or MySQL 5.1.
You may even want to swap those complaints for contributions (I know complaining is often easier). Thanks Sun!

So if you are one of the few companies yet to Open your code to the world, what are you waiting for? This Christmas give the gift of knowledge and share your achievements instead of fearing the competition and wrapping your code in a black box (don’t worry we won’t laugh at it). You’ll feel much better knowing you helped make the world a better place!

Until next year I wish you all a Very Open Source Christmas!! and as always Happy Coding!

Categories: MySQL

Great things afoot in the MySQL community

December 23, 2008 - 7:09pm

tl;dr summary - The MySQL community rocks. Percona, XtraDB, Drizzle, SSD storage, InnoDB IO scalability challenges.

For anyone who lives and dies by MySQL and InnoDB, things are finally starting to heat up and get interesting. I’ve been banging the “MySQL/InnoDB scales poorly” drums for years now, and despite having paid Enterprise licenses, I haven’t been able to get anywhere. I was pretty excited when Sun bought MySQL since their future is intrinsically tied to concurrency, but things have been pretty slow going over there this year.

But the community has finally taken up arms and is fighting the good fight. It’s (finally!) a great time to be a MySQL user because there’s been lots of recent progress. Here’re some of my favorites (and highlights of work left to do):

PERCONA

I can’t sing Percona’s praises enough. They’re probably the most knowledgeable MySQL experts out there (possibly even including Sun). Absolutely the best bang for the buck in terms of MySQL service and support - better than MySQL’s own offering. (If I had to guess why that is, I’d bet that MySQL/Sun don’t want to step on Oracle’s toes by fixing InnoDB - but >99% of what we need is related to InnoDB. Percona has no such tip-toeing limitations.) Let me quickly count the ways they’ve helped me in the last few months:

  • They knew of a super obscure configuration setting “back_log“. Have you ever heard of it? I hadn’t. But we started seeing latency on MySQL connections (up to *3 seconds*!) on systems that hadn’t changed recently (exactly 3 seconds sounded awfully suspicious, and sure enough, it was TCP retries). After going through every single kernel, network, and MySQL tuning parameter I know (and I know a lot), I finally called Percona. They dug in, investigated the system, and unearthed ‘back_log’ within an hour or two. Popped that into my configuration and boom, everything was fine again. Whew!
  • We have servers that easily exceed InnoDB’s transaction limits. Did you know InnoDB has a concurrent transaction limit of 1024? (Technically, 1024 INSERTs and 1024 UPDATEs. But INSERT … ON DUPLICATE KEY UPDATE manages to chew up one of each). I know all about it - I’ve had bugs open with MySQL Enterprise for more than 2 years on the issue. What’s more, these are low-end systems - 4 cores, 16GB of RAM - and they’re no-where near CPU or IO bound. It took MySQL months to figure out what the problem was (years, really, to figure out all the final details like the different undo logs for INSERT vs UPDATE). Their final answer? It’ll be fixed in MySQL 6. Note that 5.1 *just* went GA after years and years. On the other hand, it took Percona one weekend to diagnose the problem, and 13 days to have a preliminary patch ready to extend it to 4072 undo slots. Talk about progress! (And yes, we want Percona to release the patch to the world)
  • Solving the CPU scaling problems. These have been plaguing us for years (we have had some older four-socket systems for awhile … now with quad-core, it’s even worse), and thanks to Google and Percona, this problem is well on its way to being solved. We’re sponsoring this work and can’t wait to see what happens next.
  • XtraDB. This is the biggy. So big it deserves its own heading….

XTRADB

Oracle’s done a terrible job of supporting the community with InnoDB. The conspiracy theorists can all say “I told you so! Oracle bought them to halt MySQL progress” now - history supports them. Which is a shame - Heikki is a great guy and has done amazing work with InnoDB, but the fact remains that it wasn’t moving forward. The InnoDB plugin release was disappointing, to say the least. It addressed none of the CPU or IO scalability issues the community has been crying about for years.

Luckily, Percona finally did what everyone else has been too afraid to do - they forked InnoDB. XtraDB is their storage engine, forked from InnoDB (and then turbocharged!). We’re not running it in production yet, but we are running all of the patches that went into XtraDB and I can tell you they’re great. We’re sponsoring more XtraDB development (and yes, we made sure Percona will be contributing anything they build for us back to the community) with Percona, and I’m sure that’ll continue.

DRIZZLE

I’ve already blogged a bit about Drizzle, but it sure looks like Drizzle + XtraDB might be a match made in heaven. Drizzle can be though of as a MySQL engine re-write with an eye towards web workloads and performance, rather than features. MySQL 4.1, 5.0, and 5.1 added a lot of features that bloated the code without offering anything really useful to web-oriented workloads like ours, so the Drizzle team is ripping all that stuff back out and rethinking the approaches to the things that are being left in. Very exciting.

SSD STORAGE

The advent of “cheap enough” super-fast SSD storage is finally upon us. I’ve got Sun S7410 storage appliances in production and they’re blazingly fast. I have a very thorough review coming, but the short version is that even with NFS latencies, we’re able to do obscene write workloads to these boxes (let alone reads). 10000+ write IOPS to 10TB of mirrored, crazy durable (thanks ZFS!) storage is a dream come true. Once you mix in snapshots, clones, replication, and Analytics - well, it just doesn’t get much better than this.

(Don’t get sticker shock looking at the web pricing - no-one pays anything even remotely like that. Sign up for Startup Essentials if you can, or talk to your Sun sales rep if you can’t, and you can get them much cheaper. I nearly had a heart attack myself until I got “real” pricing. Tell them I sent you - enough Sun people read this blog, it might just help ).

STILL NEEDED…

So, all in all, there’s been an awful lot of progress this year, which is great. CPUs are finally scaling under InnoDB, and we finally have storage that isn’t bounded by physical rotation and mechanical arms. Unfortunately, great CPU scaling plus amazing IO capabilities isn’t something InnoDB digests very well. As is common in complicated systems, once you fix one bottleneck, another one elsewhere in the system crops up. This time, it’s IOPS. It was eerie reading Mark Callaghan’s post about this last night - I’d come to the exact same conclusions (from an Operations point of view rather than code-level) just yesterday.

Bottom line: Despite having ample CPU and ample IO, InnoDB isn’t capable of using the IO provided. You can bet we’ll be working with Percona, Google and Sun (read: sitting back and admiring their brilliant work while writing the occasional check and providing production workload information) to look into fixing this.

In the meantime, we’re back to the old standbys: replication and data partitioning. Yes, we’re stacking lots of MySQL instances on each S7410 to maximize both our IOPS and our budget. Fun stuff - more on that later.

UPDATE: Just occurred to me that there are plenty of *new* readers to my blog who haven’t heard me praise Google and their patches before. Mark Callaghan’s team over at Google definitely deserves a shout-out - they’ve really been a catalyst for much of this work along with Percona.

Categories: MySQL

Settling in for a Winter's Blogfest

December 23, 2008 - 7:09pm

Christmas is nearly upon the Christian and Consumerist parts of the globe. Along much of my latitude, snow is piling up in record quantities and weather warnings abound. Even in oft-green Vancouver, there is a foot or so of the white stuff accumulated in my yard and the parking lot outside my window often echos with the sounds of snow-beached cars helplessly spinning their wheels.

Usually all these things taken together would mean a series of harrowing drives to visit family, followed by lovely hours staying warm indoors, eating comfort food, retelling old stories and enjoying the company of loved ones as the year draws to its close.

However, this year our family time is coming after the holidays and I'm left with the unexpected gift of a week or so of free time.

Another unexpected thing is the current state of the Mozilla Foundation. Over the last few months of the year, we've worked particularly hard to be good little Mozilla folks. We've planned, plotted and schemed; brainstormed, drafted and designed; tacticated, calculated and strategized. We've been thinking so hard that we've had to conjugate verbs in ways you've never even heard of. From all of this, I expected Mozillaclaus to deliver some mighty fine presents: namely, a Mozilla Foundation 2009 roadmap and vision that our board approves and that all MoFo staff subscribe to. Instead, we got the early surprise of an IRS audit during the worst global financial crisis since the Great Depression. (For more commentary here, see Mitchell Baker's Sustainability in Uncertain Times blog post.)

Early surprises aside, I'm still totally jazzed about the work of the last few months. Rather than be focused on the lump of coal in our stocking and mope through the holidays, I'm going to instead spent the time blogging about some of the thinking that's been going on at the Mozilla Foundation and in my head over the last three months. Much of my blogging will focus on the ideas that I've been developing or championing, as I can't easily speak for the rest of the team when it comes to matters such as this. Also, some of these ideas are more half-baked than others - think of the posts as thought experiments or brainstorms. If you like an idea, let me know. If you hate an idea, let me know. If you can give constructive input, that is best of all.

Happy holidays folks (and remember, only two shopping days left till Christmas. ;)

Categories: MySQL

MySQL: Innodb Memory Usage Formula

December 23, 2008 - 6:43pm

I hate looking for this….

This will give you a rough idea of your innodb memory usage. I know it’s in a hundred spots, but i hate looking for it when i double check things.

innodb_buffer_pool_size
+ key_buffer_size
+ max_connections*(sort_buffer_size+read_buffer_size+binlog_cache_size +2MB)

Categories: MySQL

Linux: use lsof to find which files are open by a process

December 23, 2008 - 6:33pm

lsof works under linux and MacOS X and will help you figure out what files are open.

lsof | grep ‘mysqld’

Categories: MySQL

Cloud Computing for the White House?

December 23, 2008 - 3:25pm

Ok, Now we are talking!

Oh well, we know the Obama team is quite technology savvy and want to run the administration on the state of the art computer technologies. As an example, Obama campaign website used MySQL on the backend. 

So then, can Cloud Computing benefits lure the administration? Security and Technology experts discuss on national public radio  if Cloud Computing will work for the White House and how their computers should run.  Kevin L. Jackson further muses if the Obama Administration should use Cloud Computing.  He believes that Cloud Computing technology can indeed be used to implement the recommendations made by Center for Strategic and International Studies (CSIS)  that recently released the Commission on Cybersecurity for the 44th Presidency.  

Provide your comments to NPR.

 

Categories: MySQL

Microsoft probing SQL Server vulnerability

December 23, 2008 - 2:43pm
The vulnerability opens system to remote code execution but only affects certain versions of the database software.
Categories: MySQL

New XAMPP for Windows and Linux

December 23, 2008 - 12:11pm
Just in time for Christmas: After the usual 2-week beta test phase, we can now announce the new "final" XAMPP version for public downloading.

In both versions we updated Apache (2.2.11), MySQL (5.1.30), PHP (5.2.8) and phpMyAdmin (3.1.1). The Linux version also contains the new MySQL storage engine PBXT (1.0.07-rc).

With this version our support of PHP 4 ends and from this version on PHP 4 is no longer shipped with XAMPP. After PHP 4 was officially no longer supported since the end of 2007 this was just a matter of time.

On the other hand XAMPP now supports the brand-new MySQL 5.1 database generation. If you're upgrading from MySQL 5.0 please take a look at MySQL's official Upgrading from 5.0 to 5.1 and Monty's critical "Oops, we did it again".

Updated versions for Mac OS X and Solaris will follow shortly.

Get the downloads and more details on the specific platforms XAMPP project page.

Oh, and yes: Wish you all a Merry Christmas and a Happy New Year 2009! :)
Categories: MySQL

MySQL

Yahoo

Recent comments

About Me

Jay Janssen
Yahoo!, Inc.
jayj at yahoo dash inc dot com

MySQL
High Availability
Global Load Balancing
Failover

View Jay Janssen's LinkedIn profileView Jay Janssen's Facebook profile

User login

Friends

  • Super Geeky: Removing Subversion files from a directory with Windows Powershell
  • Sliding content from a partial height with jQuery.
  • Accessing Google Spreadsheet with PHP
  • Magento: Jump back to Payment Screen when Payment Declined
  • Laptop cooling stand – Keep your lap cool
  • Pattern Tap: We dig Jay because we got Dugg and not buried
  • How To: Build a Custom Ergonomic Computer Desk
  • New Times… New Desk
  • Google Adds Another Search Tool
  • Selling a Great Car: 1995 Honda Accord EX Wagon

Links