slave precache

Implementing a Replication precacher

I've completed a beta implementation of my take on the replication pre-cache tool... Sorry nothing to download yet, I have to get it through an internal committee at Yahoo before I can release it (and you can imagine things are kind of crazy here).  I wrote it myself because:

  1. I had it mostly done before I found out there were other versions out there
  2. I have to maintain it inside of Yahoo anyway
  3. I wanted to learn Ruby :)

It's just over 250 lines of Ruby, my new favorite language and fairly compact.  It doesn't use the Ruby Mysql library, rather just IO.popen calls to the mysql command line client.  I did this for two reasons: 

  1. I haven't figured out the "right" way to deploy ruby gems at Yahoo yet (it's complicated).

Faster way to find the next valid position in a relay log?

I'm working on an implementation of a mysql slave precache tool as described by Paul Tuckfield at last year's User conference.

I can easily tell where the SQL Thread is currently is reading from in the local relay log, but it's a harder problem to solve when I want to scan ahead in that log some number of binlog entries and find a valid position. I can simply add to the position some number of bytes, but then I have to stop and loop through calls to 'mysqlbinlog' until I get a position that doesn't throw an error.

This turns out to be a pretty expensive operation, and I'm having a problem where by the time I find a good position and can start my precacher, the slave is already past me!

Does anyone have any clever ways to find good binary log positions besides this (this is in ruby, in case you're wondering):

def mysqlbinlog_cmd( position, length )

About Me

Jay Janssen
Yahoo!, Inc.
jayj at yahoo dash inc dot com

High Availability
Global Load Balancing

View Jay Janssen's LinkedIn profileView Jay Janssen's Facebook profile