tungsten-replicator: Oracle to MySQL schema converter using ddlscan [PATCH]

The last couple of weeks we’ve been playing around with tungsten for the purpose of a full scale Oracle to MySQL replication. The newest provision feature in 2.2.1 was especially helpful – great work!

We now have a full scale replication running, and the latest “minor” obstacle we met was converting existing scheme to MySQL scheme as automatically as possible. Since this might be updated more often than first thought, we were looking for an automatic approach. Thanks to ddlscan and its features – it solved the issue for us.

The only problem with ddlscan was the inability to convert decimal precision to MySQL, hence we made som modification to the database layer of tungsten to accommodate our needs. What we essentially lacked, was the ability to convert NUMBER(14,2) to DECIMAL(14,2), and be sure that NUMBER(10, 0) could be converted to INT(11).

This is the first draft of a patch, which adds decimal precision to the Column-object, and adds the appropriate information in AbstractDatabase. I also attached a patch to the ddl-oracle-mysql Velocity-script to illustrate the use of the new feature.

See attached patch:


diff '--exclude=.svn' '--exclude=build' -ur replicator/samples/extensions/velocity/includes/ddl-oracle-mysql-table.vm sources/replicator/samples/extensions/velocity/includes/ddl-oracle-mysql-table.vm
--- replicator/samples/extensions/velocity/includes/ddl-oracle-mysql-table.vm	2014-05-06 13:55:14.000000000 +0200
+++ sources/replicator/samples/extensions/velocity/includes/ddl-oracle-mysql-table.vm	2014-05-06 15:24:26.000000000 +0200
@@ -37,7 +37,7 @@
 #else
 DECIMAL($col.getLength())##
 #end
- /* $col.getTypeDescription()($col.getLength(), ?) */##
+ /* $col.getTypeDescription()($col.getLength(), $col.getDecimalPrecision()) */##
 #elseif ( $col.getTypeDescription().startsWith("FLOAT") )
 $col.getTypeDescription()($col.getLength())##
 #if( $col.getLength() > 53 )
@@ -67,4 +67,4 @@
 #if( $colReserved ) /* WARN: changed column name because of reserved word ($col.getName()) */#end##
 #if( $foreach.hasNext ),
 #end
-#end
\ No newline at end of file
+#end
diff '--exclude=.svn' '--exclude=build' -ur replicator/src/java/com/continuent/tungsten/replicator/database/AbstractDatabase.java sources/replicator/src/java/com/continuent/tungsten/replicator/database/AbstractDatabase.java
--- replicator/src/java/com/continuent/tungsten/replicator/database/AbstractDatabase.java	2014-05-06 13:55:12.000000000 +0200
+++ sources/replicator/src/java/com/continuent/tungsten/replicator/database/AbstractDatabase.java	2014-05-06 15:19:59.000000000 +0200
@@ -986,12 +986,14 @@
                 String typeDesc = rsc.getString("TYPE_NAME").toUpperCase();
                 // Issue 798. Mimicking MySQLApplier.
                 boolean isSigned = !typeDesc.contains("UNSIGNED");
+		int decimalPrec = rsc.getInt("DECIMAL_DIGITS");
 
                 Column column = new Column(colName, colType, colLength,
                         isNotNull, valueString);
                 column.setPosition(rsc.getInt("ORDINAL_POSITION"));
                 column.setTypeDescription(typeDesc);
                 column.setSigned(isSigned);
+		column.setDecimalPrecision(decimalPrec);
                 table.AddColumn(column);
                 cm.put(column.getName(), column);
             }
diff '--exclude=.svn' '--exclude=build' -ur replicator/src/java/com/continuent/tungsten/replicator/database/Column.java sources/replicator/src/java/com/continuent/tungsten/replicator/database/Column.java
--- replicator/src/java/com/continuent/tungsten/replicator/database/Column.java	2014-05-06 13:55:12.000000000 +0200
+++ sources/replicator/src/java/com/continuent/tungsten/replicator/database/Column.java	2014-05-06 14:45:24.000000000 +0200
@@ -45,6 +45,7 @@
                                                        // NULL column
     Serializable              value;
     int                       valueInputStreamLength;
+    int decimalPrecision;
     private int               position;
     private boolean           blob;
     private String            typeDescription;
@@ -109,6 +110,16 @@
         return this.type;
     }
 
+    public int getDecimalPrecision()
+    {
+	return this.decimalPrecision;
+    }
+
+    public void setDecimalPrecision(int precision)
+    {
+	this.decimalPrecision = precision;
+    }
+
     public long getLength()
     {
         return this.length;

applesmc read fail on mid-2012 Macbooks

The error is simple “applesmc: read arg fail” – this happens when trying to modprobe the applesmc module.

[   46.584969] applesmc: LSSB: read arg fail
[   46.672745] applesmc: LSSB: read arg fail
[   46.759992] applesmc: : read arg fail
[   46.847830] applesmc: : read arg fail
[   46.935599] applesmc: : read arg fail
[   47.023427] applesmc: : read arg fail
[   47.111244] applesmc: : read arg fail
[   47.199101] applesmc: : read arg fail

So I’ve been hacking around and reading forum threads when this genius over at Ubuntu forums came up with the solution – really simple.

The solution is a patch which actually only modifies one line of code – the constant telling the read from SMC to wait for feedback.

static int __wait_status(u8 val)
{
    int us;
    val = val & APPLESMC_STATUS_MASK;
 
    for (us = APPLESMC_MIN_WAIT; us < APPLESMC_MAX_WAIT; us <<= 1) {
        udelay(us);
        if ((inb(APPLESMC_CMD_PORT) & APPLESMC_STATUS_MASK) == val)
            return 0;
    }
 
    return -EIO;
}

The constant to be changed is the APPLESMC_MIN_WAIT.

#define APPLESMC_MIN_WAIT     0x0040

is changed to

#define APPLESMC_MIN_WAIT     0x0010

This should take care of your problems!

Compiling the new module on Ubunutu

On Ubuntu systems you can get the kernel source by typing

sudo apt-get install linux-source

You can find the source in /usr/src. Uncompress it!

When you’ve applied the patch

index 0162f55..f41585e 100644
--- a/drivers/hwmon/applesmc.c
+++ b/drivers/hwmon/applesmc.c
@@ -54,7 +54,7 @@
#define APPLESMC_MAX_DATA_LENGTH 32
/* wait up to 32 ms for a status change. */
-#define APPLESMC_MIN_WAIT	0x0040
+#define APPLESMC_MIN_WAIT	0x0010
#define APPLESMC_MAX_WAIT	0x8000
 
#define APPLESMC_STATUS_MASK	0x0f
--
1.7.11.1

Then simply run

cd /usr/src/linux-source-x.x.x/linux-source-x-x-x/drivers/hwmon/
sudo make -C /lib/modules/$(uname -r)/build M=$(pwd) applesmc.ko
sudo insmod applesmc.ko
dmesg | grep applesmc

`dmesg | grep applesmc` should read something like
[ 70.969188] applesmc: key=375 fan=1 temp=22 acc=1 lux=2 kbd=1

Hope this works for you! :)

How to get system-information with dmidecode

Ever wondered what kind of RAM your system supports? Or maybe you can’t remember your motherboard brand or model? Well, luck you, linux has a perfect tool for finding this information.

First, let’s have a look at what we actually can find out:

Type Information
────────────────────────────────────────
0 BIOS
1 System
2 Base Board
3 Chassis
4 Processor
5 Memory Controller
6 Memory Module
7 Cache
8 Port Connector
9 System Slots
10 On Board Devices
11 OEM Strings
12 System Configuration Options
13 BIOS Language
14 Group Associations
15 System Event Log
16 Physical Memory Array
17 Memory Device
18 32-bit Memory Error
19 Memory Array Mapped Address
20 Memory Device Mapped Address
21 Built-in Pointing Device
22 Portable Battery
23 System Reset
24 Hardware Security
25 System Power Controls
26 Voltage Probe
27 Cooling Device
28 Temperature Probe
29 Electrical Current Probe
30 Out-of-band Remote Access
31 Boot Integrity Services
32 System Boot
33 64-bit Memory Error
34 Management Device
35 Management Device Component
36 Management Device Threshold Data
37 Memory Channel
38 IPMI Device
39 Power Supply

Continue reading How to get system-information with dmidecode

Howto: Setup Ubuntu on MacBook Air without a SuperDrive

I looked all over the net for a guide on how to setup Ubuntu on a MacBook Air without buying the SuperDrive, but couldn’t find anything. So I hade to do it the hard way, figure it out myself!

What you need is:

  • MacBook Air
  • External drive of some kind with enough space for a Ubuntu install disk
  • Ubuntu Hardy Heron image

First step

You need to start out by creating a bootable Ubuntu disk. I’ll do a quick guide on that, if it doesn’t work, theres a lot of guides out there that could help you out. Forget it! I’ll just post a link.

https://help.ubuntu.com/community/Installation/FromUSBStick

I’de suggest to use the isotostick.sh script to do it, worked good for me.

Second step

So you’ve got a disk or a stick with a Ubuntu Hardy Heron image on it, whats next?

First we need to partition the harddrive on the MacBook Air. I suggest using the “Disk Utility” in OSX, works quick and is easy. What you need to do, is use the disk utility to create TWO additional partitions. HFS+ allows resizing, so the data on your disk will not be lost.

Just choose your main harddrive, and then the tab called “Partition“.

Then add 2 new partitions
Continue reading Howto: Setup Ubuntu on MacBook Air without a SuperDrive