{
"_id" : ObjectId("4bd8ae97c47016442af4a580"),
"customerid" : 99999,
"name" : "Foo Sushi Inc",
"type" : "Good",
"since" : "12/12/2001",
"addresses" : [{
"address" : "4821 Big Street",
"city" : "Stone",
"state" : "IL",
"country" : "USA"
},
{ "address" : "1248 Barlow Ln",
"city" : "Hedgestone",
"country" : "UK"
}
],
"emails" : [
{"email" : "foousa@sushi.com"},
{"email" : "foouk@sushi.com"}
],
"phones" : [
{"phone" : "773-7777-7777"},
{"phone" : "020-6666-6666"}
]
}
So Instead of 1 Row for customer, 2 rows for address, phone and email each, you get one Customer document. If you want to query for customers in USA. Using RDBMS you would do
SELECT customer.name FROM customer, address WHERE customer.customerid = address.customerid AND address.country="USA"The same query in mongo would look like
db.customers.find({"addresses.country":"USA"},{"name":true})
where customers is the collection in which we store our customers.]]>
./mongod --dbpath=/user/data/dbis all you need, where /user/data/db is the path where you want mongo to create its data files. There are many other options that you can use to customize the mongo instance. Each mongo instance has databases and each database has many collections, mapping back to oracle, mongo database is a oracle schema and mongo collection is a oracle table, The difference is, each collection can hold any type of object, basically every row can be different. An example connection to the database using java looks like this
Mongo mongo = new Mongo("localhost");
db = mongo.getDB("mydatabase");
If the "mydatabase" does not exist, it will be created. When you want to put objects in the database, you need to have a collection which holds the objects.
users = db.getCollection("applicationusers");
if the "applicationusers" collection does not exist, it will be created, at this point you are ready to put objects into the collection.
BasicDBObject userDocument = new BasicDBObject();
userDocument.put("name", "jack");
userDocument.put("type", "super");
users.insert(userDocument);
You create a document by using the BasicDBObject and put attribute names and their values, in the above example "name" is the attribute and "jack" is the value, the users.insert takes the document and inserts it into the collection "users". At this point you have a JSON object put into the database.
You can query for the object using the mongo query tool or the rest full api mongo provides using the flag --rest, when you start mongodb, visiting http://127.0.0.1:28017/mydatabase/users/ should give you
{
"offset" : 0,
"rows": [
{ "_id" : { "$oid" : "4bc9157e201f254d204226bf" }, "name" : "jack", "type" : "super" }
],
"total_rows" : 1 ,
"query" : {} ,
"millis" : 0
}
Every object you insert, gets a auto generated id, more about update, delete and complex objects in next blog post.]]>
Since data is moved/changed from a source database to destination database, we can write sql which should provide results for the types of tests you want to perform, for example: write a sql to give us number of customers, write a sql to give us account balance for a specific account.
These sqls can be run on your source database as well as your destination database and the results can be compared programmatically, providing us an easy way to compare the state of the database before and after conversion/migration. This testing can be run through a CI engine to make it a regression test suite.
Here is an example implementation using ruby,
We have two databases SOURCE and DESTINATION and two sql files names source.sql and destination.sql. The ruby program picks up sql from these two files and runs them against their database i.e. sql from source.sql is run against the SOURCE database and sql from destination.sql is run against DESTINATION database. The results of both of those sqls is compared and an failure is raised when the results do not match.
results
statement = get_sql_statement_to_execute
begin
source_statement = statement[0]
destination_statement = statement[1]
source_rows = exec_sql_in_source_return_rows(source_statement)
destination_rows = exec_sql_in_destination_return_rows(destination_statement)
result = compare_rows(source_rows, destination_rows, destination_statement, source_statement)
results << result
rescue
Log.log("Could not process: "+statement)
end
if (results.size > 0)
Log.log("Results do not match in source and destination")
end
The sample ruby code above shows how the solution can be implemented, thus enabling automation of database conversion/migration testing
]]>Lets say you want to insert 10 rows, using the insert one row at a time would be 10 trips to the database.
def save_accounts(accounts)
stmt = $connection.parse "INSERT INTO account (accountid,name) values (:account_id,:name)"
accounts.each do |account|
stmt.bind_param(:account_id, account[0], Float)
stmt.bind_param(:name, account[1], String)
stmt.exec
end
$connection.commit
end
Using the array bind feature, its actually just one trip to the database (off course depends on the array size you are going to bind, but you get the picture, it reduces database trips)
def save_accounts(account_ids, account_names)
stmt = $connection.parse "INSERT INTO account (accountid,name) values (:account_id,:name)"
stmt.max_array_size= account_ids.size
stmt.bind_param_array(:account_id, account_ids)
stmt.bind_param_array(:name, account_names)
stmt.exec_array
$connection.commit
end
We saw a 100% improvement in performance by changing the way we bind the variables in just one place. Looks like a feature to look out for.]]>
SELECT ... FROM ORDER WHERE CUSTOMERID = 1000;now if there is no index on ORDER.CUSTOMERID, the database will have to do a full Table scan which is very expensive in terms of IO and resources, imagine customerId being used in lots of tables, the problem just multiplies significantly. In an multiuser scenario, this will lead to deadlocks, since the same tables are being read and locks being applied to find dependend children. Introducing an index on all the columns that are foreign key referenced helps a lot in this case.]]>
Lets say the two databases in question are local and remote, so the sql to create the materialized view to load immediately and refresh everyday is
CREATE MATERIALIZED VIEW MV_CUSTOMERBALANCE BUILD IMMEDIATE REFRESH FORCE START WITH ROUND(SYSDATE) + 23/24 NEXT SYSDATE + 1 AS SELECT customer.name , account.balance, accounttype.name FROM customer , account@remotedb account, accounttype@remotedb accounttype WHERE customer.id = account.customerid AND account.accounttyppeid = accounttype.id /Oracle started to complain when creating the above materialized view issuing an error ORA-00942: table or view does not exist, but the SQL without the create materialized view command ran fine giving the expected results.
SELECT customer.name , account.balance, accounttype.name FROM customer , account@remotedb account, accounttype@remotedb accounttype WHERE customer.id = account.customerid AND account.accounttyppeid = accounttype.id /After some searching around and experimenting I found, in the create materialized view statement the database link name can be used only once, which meant we can only use the "remotedb" name once, we got around this restriction by creating two database links to the remote database as REMOTEACCOUNT and REMOTEACCOUNTTYPE and using them in the creation of the materialized view as shown below.
CREATE MATERIALIZED VIEW MV_CUSTOMERBALANCE BUILD IMMEDIATE REFRESH FORCE START WITH ROUND(SYSDATE) + 23/24 NEXT SYSDATE + 1 AS SELECT customer.name , account.balance, accounttype.name FROM customer , account@remoteaccount account, accounttype@remoteaccounttype accounttype WHERE customer.id = account.customerid AND account.accounttyppeid = accounttype.id /]]>
I go regularly to this store to get Horizon organic milk for my family, about 60% of the time I see milk I need NOT in stock, okay I can live with that, may be lots of folks are buying organic milk, but not when it happens frequently, especially when the store knows how much milk was ordered (or supplied from the warehouse) and how much milk was sold, the store should be able to figure out that organic milk gets sold out pretty fast, putting my Business Intelligence (BI) hat on, I think the store should be able to predict when they are going to run out of organic milk ( for that matter any product), its especially frustrating when they have all the data they need to get it done.
One more non usage of data that really makes me red is, when the organic milk in the store is already expired (past the sell by date). I mean how hard is it for someone to generate a list of all the products that expire today and ask the store associates to remove them from the shelves by the end of the day, especially when its edible items.
For example.
Lets assume there are two tables Parent(ParentID) and Child(ChildID)
SESSION_A >create table parent (parentId number(10));
Table created.
SESSION_A >create table child (childId number(10));
Table created.
SESSION_A >insert into parent values (100);
1 row created.
SESSION_A >insert into child values (200);
1 row created.
SESSION_A >commit;
Commit complete.
SESSION_A >select * from parent;
PARENTID
----------
100
SESSION_A >select * from child;
CHILDID
----------
200
SESSION_A >
Now lets create a situation where a deadlock happens. There are two sessions connected to the same database and same user, SESSION_A and SESSION_B are the two sessions in question.
SESSION_A >update parent set parentid = 1000 where parentid=100;
1 row updated.
SESSION_B >update child set childid = 2000 where childid = 200;
1 row updated.
SESSION_B >update parent set parentid = 2001 where parentid=100;
--Waiting For Lock on Row in Parent Table, held by SESSION_A
SESSION_A >update child set childid = 1001 where childid = 200;
update child set childid = 1001 where childid = 200
*
ERROR at line 1:
ORA-00060: deadlock detected while waiting for resource
--SESSION_A requesting lock on row, held by SESSION_B causing deadlock.
SESSION_A >
After you get the ORA-00060 error the statement update child set childid = 1001 where childid = 200; is rolled back.. but SESSION_B is still waiting for the lock on the Parent table to be released.
So when your application get the ORA-00060 or any deadlock exception in any other database, explicitly rollback your transaction (not just the current statement) so that all the changes made in the transaction and all the locks held by the transaction are released.
]]>This is especially nice since the Oracle for Mac was the most voted requirement on mix.oracle.com
]]>If you upgrade to Oracle 11g (I know lot of you are waiting for 11gR2), you will find that passwords are case sensitive. Here is an example of case sensitive passwords.
c:\Software>sqlplus bddd/bddd@dosa SQL*Plus: Release 11.1.0.6.0 - Production on Wed May 6 15:17:43 2009 Copyright (c) 1982, 2007, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - Production With the Partitioning and OLAP options BDDD@dosa >
Lets try to connect with a upper case password
c:\Software>sqlplus bddd/BDDD@dosa SQL*Plus: Release 11.1.0.6.0 - Production on Wed May 6 15:19:25 2009 Copyright (c) 1982, 2007, Oracle. All rights reserved. ERROR: ORA-01017: invalid username/password; logon denied Enter user-name:
So what does this mean to apps running with 10g, that get ported to run with 11g. Make sure that the password set in the properties files is of the correct case.
You can also revert to 10g behavior by changing sec_case_sensitive_logon parameter to FALSE, since its TRUE by default.
alter system set sec_case_sensitive_logon=FALSE;]]>
COLUMN OBJECT_NAME FORMAT A30
COLUMN STATUS FORMAT A15
spool objects.log
CREATE TABLE FOO (ID NUMBER(10), NAME VARCHAR2(30));
CREATE OR REPLACE TRIGGER TRIG_FOO
BEFORE INSERT OR UPDATE
ON FOO
REFERENCING OLD AS OLD NEW AS NEW
FOR EACH ROW
BEGIN
IF :NEW.name IS NULL THEN
:NEW.name := 'NOT AVAILABLE';
END IF;
END;
/
CREATE OR REPLACE FUNCTION FUNCTION_GET_NAME_FOR_FOOID(inFooId number)
RETURN VARCHAR2
IS
fooName VARCHAR2(30);
BEGIN
BEGIN
SELECT name INTO fooName FROM foo WHERE id = inFooId ;
EXCEPTION
WHEN NO_DATA_FOUND THEN
RETURN 'NOT FOUND';
END;
RETURN fooName;
END;
/
SELECT OBJECT_NAME,STATUS FROM USER_OBJECTS WHERE STATUS='INVALID';
ALTER TABLE FOO ADD ( DESCRIPTION VARCHAR2(100));
SELECT OBJECT_NAME,STATUS FROM USER_OBJECTS WHERE STATUS='INVALID';
spool off
To get the objects back to VALID status, all that needs to be done is
ALTER TRIGGER TRIG_FOO COMPILE; ALTER FUNCTION FUNCTION_GET_NAME_FOR_FOOID COMPILE; SELECT OBJECT_NAME,STATUS FROM USER_OBJECTS WHERE STATUS='INVALID';]]>
CREATE TABLE FOO (PREFERRED_TIME DATE NULL);
INSERT INTO FOO (TO_DATE('11:34','HH24:MI'));
oracle automatically sets the date to the first day of the current month. so when you do a select from the FOO table the data would be
SELECT TO_CHAR(PREFERRED_TIME, 'dd-mon-yyyy hh24:mi:ss') from FOO; ----------------------- 01-feb-2009 11:24:00
on the client side you will have to know to ignore the date component, this is possible area for confusion in future, since I could have many preferences over different months and my preferred time would have different date components, ideally I would just want the time, otherwise I would think having a time component and the date being a constant known value like 01/01/0001. We could achieve this using a BEFORE INSERT/UPDATE trigger which keeps the time component but updates the date component to a known constant value or you can also use your OR mapping layer like hibernate to set the value as such. If you are using hibernate, you can map the filed using the java.sql.Time object and the date is automatically set to 01/01/1970. SqlServer 2008 seems has a Time data type.
]]>1. Make the stored procedure handle one object/table only and not multiple objects or tables.
2. Do not commit open transactions inside the stored procedures.
3. Do not do business logic in stored procedures.
4. If they are straight CRUD stored procedures, see if you can you generate the stored procedure code using some metadata?
5. Make sure creation and execution of the stored procedures is part of your Continuous Integration build and developer build.
6. Make sure stored procedures (or the metadata used to generate them) is under Version Control, have seen many problems when the stored procedure version does not match application code version
7. Develop against the production stack database.
8. Make sure exceptions thrown by the database are passed back to the application.