Data | Architecture and Data Blog

Two factor authentication to authorize credit"

Posted by Pramod Sadalage on Wednesday, July 13, 2016

Perfectly good data.. wasted

Okay this is kind of a rant, maybe I’m too picky or just that I hate to see perfectly good data not being used. This is how it goes..

I go regularly to this store to get Horizon organic milk for my family, about 60% of the time I see milk I need NOT in stock, okay I can live with that, may be lots of folks are buying organic milk, but not when it happens frequently, especially when the store knows how much milk was ordered (or supplied from the warehouse) and how much milk was sold, the store should be able to figure out that organic milk gets sold out pretty fast, putting my Business Intelligence (BI) hat on, I think the store should be able to predict when they are going to run out of organic milk ( for that matter any product), its especially frustrating when they have all the data they need to get it done.

Posted by Pramod Sadalage on Wednesday, August 5, 2009

Long Running Data Migrations during Database Refactorings

how to manage long running data migrations

When you are refactoring large databases, you will have certain tables that have millions of rows, so lets say we are doing the Move Column refactoring, moving the TaxAmount column from Charge table which has millions of rows to TaxCharge table. Create the TaxAmount column in the TaxCharge table. Then have to move the data from the TaxAmount column in the Charge table to the TaxAmount column you created in the TaxCharge table.

Posted by Pramod Sadalage on Friday, June 29, 2007

Data Quality and Code Quality

Code quality and data quality are related

Recently we had peculiar problem. Some of the data in the database was not being created in a proper fashion. Once we found that out we fixed the problem in the application. The customer still had the perception that the code is still broken, because the fixed code was now interacting with the data that was broken (since it was created much earlier by code that was broken). Data has a life of its own (more on this later)

Posted by Pramod Sadalage on Thursday, February 15, 2007

Nulls need special love

The following SELECT statement in code

stmt = DB.prepare("select id,name,state,zip " +
        "from customer " +
        "where " +
        "phone = ? " +
        "and active = ?");
stmt.setString(1, customerPhone);
stmt.setBoolean(2, isActive);
stmt.execute();

where customerPhone and isActive are values you would pass in to the SELECT before its executed. Everything is fine when one day the value passed for customerPhone is NULL. For a database (Oracle is what I know most) a NULL will never be equal to NULL , the SELECT will not return rows where the customer.phone is NULL, leading to wrong results. The SELECT will have to be changed to