Thursday, May 10, 2012

Difference between LGWR SYNC and ASYNC in Oracle DataGuard


Oracle Data Guard redo log transport offers synchronous log transport mode (LogXptMode = 'SYNC') or asynchronous log transport mode (LogXptMode = 'ASYNC').  The difference is all about when the COMMIT happens . 

LogXptMode = ('SYNC'):  As the name implies, SYNC mode synchronizes the primary with the standby database and all DML on the primary server will NOT be committed until the logs have been successfully transported to the standby servers.  The synchronous log transport mode is required for the Maximum Protection and Maximum Availability data protection modes.
LogXptMode = ('ASYNC'): Conversely, asynchronous mode (ASYNC) allows updates (DML) to be committed on the primary server before the log file arrives on the standby servers.  The asynchronous log transport mode is required for the Maximum Performance data protection mode.

There is a very good post witten by "Shawn Kelley" related to Sync and Async in dataguard .

LGWR is an attribute of the LOG_ARCHIVE_DEST_n parameter which is used to specify the network transmission mode. Specifying the SYNC attribute (which is the default), tells the LGWR process to synchronously archive to the local online redo log files at the same time it transmits redo data to archival destinations. Specifically, the SYNC atrribute performs all network I/O synchornously in conjunction with each write operation to the online redo log file. Transactions are not committed on the primary database until the redo data necessary to recover the transactions is received by the destination.

The ASYNC attribute perfoms all network I/O asynchronously and control is returned to the executing application or user immediately. When this attribute is specified, the LGWR process archives to the local online redo log file and submits the network I/O request to the network server (LNSn process for that destination, and the LGWR process continues processing the next request without waiting for the network I/O to complete.

What happens if the network between the Primary and Standby [database] is lost with LGWR SYNC and ASYNC ? or What happens if the standby database is shutdown with LGWR SYNC and ASYNC?
This is dependent upon the database mode we have set. If we have set Maximum Protection, we have chosen a configuration that guarantees that no data loss will occur. We have set this up by specifying the LWGR, SYNC, and AFFIRM attributes of the LOG_ARCHIVE_DEST_n parameter for at least one standby database. This mode provides the highest level of data protection possible and to achieve this the redo data needed to recover each transaction must be written to both the local online redo log and the standby redo log on at least one standby database before the transaction commits. To ensure data loss cannot occur, the primary database shuts down if a fault (such as the network going down) prevents it from writing its redo stream to at least one remote standby redo log.

If  we have set the Maximum Availability mode, we have chosen a configuration that provides the highest level of data protection that is possible without compromising the availablity of the primary database. Like the maximum database does not shut down if a fault prevents it from writing its redo stream to a remote standby redo log. Instead, the primary database operates in maximum performance mode until the fault is corrected and all gaps in redo log files are resolved. When all gaps are resolved, the primary database automatically resumes operating in maximum availabitly mode. This guarantees that no data loss will occur if the primary database fails, but only if a second fault does not complete set of redo data being sent from the primary database to at least one standby database.

If we have set the Maximum Performance mode (the default), we have chosen a mode that provides the highest level of data protection that is possible without affecting the performance of the primary database. This is accomplished by allowing a transaction to commit as soon as the redo data needed to recover the transaction is written to the local online redo log. The primary database's redo data stream is also written to at least one standby database, bu that the redo stream is written asynchronously with respect to the commitment of the transactions that create the redo data.

The maximum performance mode enables us to either set the LGWR and AYSNC attributes, or set the ARCH attribute on the LOG_ARCHIVE_DEST_n parameter for the standby database destination. If the primary database fails, we can reduce the amount of data that is not received on the standby destination by setting the LGWR and ASYNC attributes.

If LGWR SYNC or ASYNC is deployed, what process(es) bring(s) the standby database back into sync with the primary [database] if the network is lost and is then restored? How does it do it?

Again, this is dependent upon the mode we have chosen for our database. The LGWR process (and possibly the LNSn process if we have multiple standby databases) is responsible for closing the gap.

My biggest question is, when the network to the standby is lost with SYNC or ASYNC, where is the information queued and how is it retransmitted once the network has been re-established?
This implies that our database has been set to either maximum availability or maximum performance mode. We cannot use the ASYNC attribute with maximum protection mode. The information is queued in the local online redo log and the LGWR (and the LNSn) process will transmit the data to the standby database's online redo log file to close the gap once the network connectivity has been re-established

Gap recovery is handled through the polling mechanism. For physical and logical standby databases, Oracle Change Data Capture, and Oracle Streams, Data Guard performs gap detection and resolution by automatically retrieving missing archived redo log files from the primary database. No extra configuration settings are required to poll the standby database(s) to detect any gaps or to resolve the gaps.

The important consideration here is that automatic gap recovery is contigent upon the availablity of the primary database. If the primary database is not available and we have a configuration with mulitple physical standby databases, we can set up additional initialization parameters so that the Redo Apply can resolve archive gaps from another standby database.

It is possible to manually determine if a gap exists and to resolve those archive gaps. To manually determine if a gap exists, query the V$ARCHIVE_GAP view on our physical standby database. If a gap is found, we will then need to locate the archived log files on our primary database, copy them to our standby database, and register them.


Enjoy       :) 

Tuesday, May 8, 2012

Insert an Image File into Oracle Database


Photographs and pictures and Oracle BLOB data are easy to add to a Oracle table. There are two ways to load BLOBs and CLOBs into the database. The first method uses PL/SQL and the DBMS_LOB package and the BFILE datatype to transfer external LOB files into the database internal LOB structures. The second uses the Oracle Call Interface (OCI) to perform the same function. Here, we will use the first method . For inserting an image, we follow the folowing steps : 


Step 1 : First, we  need to create a directory on the database (which is mapped to a directory in the server's filesystem). The user must be granted the create any directory  privilege.
SQL>create directory photo_dir as 'c:\photo_dir' ; 
Directory created.

Step 2 : Then we need to create a table which is used by procedure to insert the image in our table . Here we have to use a BLOB to insert the image .
SQL> create table temp_photo 
 (
 ID    NUMBER(3)     NOT NULL,
 PHOTO_NAME      VARCHAR2(50),
 PHOTO    BLOB
 );
Table created.

Step 3 : Now let's write the procedure to insert the image in the table above.

SQL> create or replace PROCEDURE load_file (
 p_id number,
 p_photo_name in varchar2) IS
 src_file BFILE;
 dst_file BLOB;
 lgh_file BINARY_INTEGER;
 BEGIN
  src_file := bfilename('PHOTO_DIR', p_photo_name);
  -- insert a NULL record to lock
  INSERT INTO temp_photo
  (id, photo_name, photo)
  VALUES
  (p_id , p_photo_name ,EMPTY_BLOB())
  RETURNING photo INTO dst_file;
  -- lock record
  SELECT photo 
  INTO dst_file
  FROM temp_photo
  WHERE id = p_id
  AND photo_name = p_photo_name
  FOR UPDATE;
  -- open the file
  dbms_lob.fileopen(src_file, dbms_lob.file_readonly);
  -- determine length
  lgh_file := dbms_lob.getlength(src_file);
  -- read the file
  dbms_lob.loadfromfile(dst_file, src_file, lgh_file);
  -- update the blob field
  UPDATE temp_photo 
  SET photo = dst_file
  WHERE id = p_id
  AND photo_name = p_photo_name;
  -- close file
  dbms_lob.fileclose(src_file); 
 END load_file;
/


Step 4 :  We can test it from SQL*Plus  
SQL> execute load_file(1,'rdht.jpg') ; 


Note : Remember that the file rdht.jpg should exist in the server's 'c:\photo_dir' directory .




Enjoy    :-)



Saturday, April 28, 2012

Understanding Indexes Concept


Indexes plays and crucial role in the performance tunning of a database . It is very important to know how the index  work i.e, how indexes fetches the data's from a tables . There is a very good post by  rleishman on the working of indexes . Let's have a look . 

What is an Index ? 
An index is a schema object that contains an entry for each value that appears in the indexed column(s) of the table or cluster and provides direct, fast access to rows. It is just as the index in this manual helps us to locate information faster than if there were no index, an Oracle Database index provides a faster access path to table data . 

Blocks 
First we need to understand a block. A block - or page for Microsoft boffins - is the smallest unit of disk that Oracle will read or write. All data in Oracle - tables, indexes, clusters - is stored in blocks. The block size is configurable for any given database but is usually one of 4Kb, 8Kb, 16Kb, or 32Kb. Rows in a table are usually much smaller than this, so many rows will generally fit into a single block. So we never read "just one row"; we will always read the entire block and ignore the rows we don't need. Minimising this wastage is one of the fundamentals of Oracle Performance Tuning.

Oracle uses two different index architectures: b-Tree indexes and bitmap indexes. Cluster indexes, bitmap join indexes, function-based indexes, reverse key indexes and text indexes are all just variations on the two main types. b-Tree is the "normal" index .

The "-Tree" in b-Tree 
A b-Tree index is a data structure in the form of a tree - no surprises there - but it is a tree of database blocks, not rows. Imagine the leaf blocks of the index as the pages of a phone book .  Each page in the book (leaf block in the index) contains many entries, which consist of a name (indexed column value) and an address (ROWID) that tells us the physical location of the telephone (row in the table).
The names on each page are sorted, and the pages - when sorted correctly - contain a complete sorted list of every name and address

A sorted list in a phone book is fine for humans, beacuse we have mastered "the flick" - the ability to fan through the book looking for the page that will contain our target without reading the entire page. When we flick through the phone book, we are just reading the first name on each page, which is usually in a larger font in the page header. Oracle cannot read a single name (row) and ignore the reset of the page (block); it needs to read the entire block.

If we had no thumbs, we may find it convenient to create a separate ordered list containing the first name on each page of the phone book along with the page number. This is how the branch-blocks of an index work; a reduced list that contains the first row of each block plus the address of that block. In a large phone book, this reduced list containing one entry per page will still cover many pages, so the process is repeated, creating the next level up in the index, and so on until we are left with a single page: the root of the tree.

For example : 
To find the name Gallileo in this b-Tree phone book, we:
=> Read page 1. This tells us that page 6 starts with Fermat and that page 7 starts with Hawking.
=> Read page 6. This tells us that page 350 starts with Fyshe and that page 351 starts with Garibaldi.
=> Read page 350, which is a leaf block; we find Gallileo's address and phone number.
=> That's it; 3 blocks to find a specific row in a million row table. In reality, index blocks often fit 100 or more rows, so b-Trees are typically quite shallow. I have never seen an index with more than 5 levels. Curious? Try this:

SQL> select index_name,  blevel+1  from  user_indexes  order  by  2 ;
user_indexes.blevel is the number of branch levels. Always add 1 to include the leaf level; this tells us the number of blocks a unique index scan must read to reach the leaf-block. If we're really, really, insatiably curious; try this in SQL*Plus:

SQL> accept   index_name  prompt   "Index Name: " 
SQL> alter session set tracefile_identifier='&index_name' ; 
SQL> column object_id new_value object_id
SQL> select  object_id  from user_objects where object_type = 'INDEX'  and  object_name=upper('&index_name');
SQL> alter session set events 'Immediate trace name treedump level &object_id';
SQL> alter session set tracefile identifier="" ;
SQL> show parameter user_dump_dest 


Give the name of an index on a smallish table (because this will create a BIG file). Now, on the Oracle server, go to the directory shown by the final SHOW PARAMETER user_dump_dest command and find the trace file - the file name will contain the index name. Here is a sample:

---- begin tree dump
branch: 0x68066c8 109078216 (0: nrow: 325, level: 1)
   leaf: 0x68066c9 109078217 (-1: nrow: 694 rrow: 694)
   leaf: 0x68066ca 109078218 (0: nrow: 693 rrow: 693)
   leaf: 0x68066cb 109078219 (1: nrow: 693 rrow: 693)
   leaf: 0x68066cc 109078220 (2: nrow: 693 rrow: 693)
   leaf: 0x68066cd 109078221 (3: nrow: 693 rrow: 693)
   ...
   ...
   leaf: 0x68069cf 109078991 (320: nrow: 763 rrow: 763)
   leaf: 0x68069d0 109078992 (321: nrow: 761 rrow: 761)
   leaf: 0x68069d1 109078993 (322: nrow: 798 rrow: 798)
   leaf: 0x68069d2 109078994 (323: nrow: 807 rrow: 807)
----- end tree dump
This index has only a root branch with 323 leaf nodes. Each leaf node contains a variable number of index entries up to 807! A deeper index would be more interesting, but it would take a while to dump.

"B"  is  for...
Contrary to popular belief, b is not for binary; it's balanced.
As we insert new rows into the table, new rows are inserted into index leaf blocks. When a leaf block is full, another insert will cause the block to be split into two blocks, which means an entry for the new block must be added to the parent branch-block. If the branch-block is also full, it too is split. The process propagates back up the tree until the parent of split has space for one more entry, or the root is reached. A new root is created if the root node splits. Staggeringly, this process ensures that every branch will be the same length. 

How are Indexes used ?
Indexes have three main uses:

  • To quickly find specific rows by avoiding a Full Table Scan

We've already seen above how a Unique Scan works. Using the phone book metaphor, it's not hard to understand how a Range Scan works in much the same way to find all people named "Gallileo", or all of the names alphabetically between "Smith" and "Smythe". Range Scans can occur when we use >, <, LIKE, or BETWEEN in a WHERE clause. A range scan will find the first row in the range using the same technique as the Unique Scan, but will then keep reading the index up to the end of the range. It is OK if the range covers many blocks.
  • To avoid a table access altogether

If all we wanted to do when looking up Gallileo in the phone book was to find his address or phone number, the job would be done. However if we wanted to know his date of birth, we'd have to phone and ask. This takes time. If it was something that we needed all the time, like an email address, we could save time by adding it to the phone book.
Oracle does the same thing. If the information is in the index, then it doesn't bother to read the table. It is a reasonably common technique to add columns to an index, not because they will be used as part of the index scan, but because they save a table access. In fact, Oracle may even perform a Fast Full Scan of an index that it cannot use in a Range or Unique scan just to avoid a table access.
  • To avoid a sort

This one is not so well known, largely because it is so poorly documented (and in many cases, unpredicatably implemented by the Optimizer as well). Oracle performs a sort for many reasons: ORDER BY, GROUP BY, DISTINCT, Set operations (eg. UNION), Sort-Merge Joins, uncorrelated IN-subqueries, Analytic Functions). If a sort operation requires rows in the same order as the index, then Oracle may read the table rows via the index. A sort operation is not necessary since the rows are returned in sorted order.

Despite all of the instances listed above where a sort is performed, I have only seen three cases where a sort is actually avoided.

1. GROUP BY : 


SQL> select src_sys, sum(actl_expns_amt), count(*)  from ef_actl_expns
           where src_sys = 'CDW'   and actl_expns_amt > 0  
           group by src_sys ; 
-----------------------------------------------------------------------------------------
| Id   |      Operation                                               |     Name             |
----------------------------------------------------------------------------------------
|   0  | SELECT STATEMENT                                     |                           |
|   1  |  SORT GROUP BY NOSORT  <-------           |                           |
|*  2 |   TABLE ACCESS BY GLOBAL INDEX ROWID | EF_ACTL_EXPNS |
|*  3 |    INDEX RANGE SCAN                                 | EF_AEXP_PK       |
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
----------------------------------------------------------------
   2 - filter("ACTL_EXPNS_AMT">0)
   3 - access("SRC_SYS"='CDW')
Note the NOSORT qualifier in Step 1.

2. ORDER BY : 


SQL> select *  from ef_actl_expns
          where src_sys = 'CDW' and actl_expns_amt > 0
          order by src_sys 
----------------------------------------------------------------------------------------
| Id   | Operation                                                     |     Name            |
----------------------------------------------------------------------------------------
|   0  | SELECT STATEMENT                                     |                           |
|*  1 |  TABLE ACCESS BY GLOBAL INDEX ROWID   | EF_ACTL_EXPNS|
|*  2 |   INDEX RANGE SCAN                                   | EF_AEXP_PK      |
----------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter("ACTL_EXPNS_AMT">0)
   2 - access("SRC_SYS"='CDW')

Note that there is no SORT operation, despite the ORDER BY clause. Compare this to the following:

SQL>  select * from ef_actl_expns
            where src_sys = 'CDW'  and actl_expns_amt > 0
            order by actl_expns_amt ; 
---------------------------------------------------------------------------------------------
| Id  | Operation                                                      |         Name          |
---------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                                       |                            |
|   1 |  SORT ORDER BY                                            |                            |
|*  2 |   TABLE ACCESS BY GLOBAL INDEX ROWID   | EF_ACTL_EXPNS |
|*  3 |    INDEX RANGE SCAN                                   | EF_AEXP_PK       |
----------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   2 - filter("ACTL_EXPNS_AMT">0)
   3 - access("SRC_SYS"='CDW')

3. DISTINCT : 


SQL> select distinct src_sys  from ef_actl_expns
           where src_sys = 'CDW'  and actl_expns_amt > 0 ; 
-----------------------------------------------------------------------------------------------
| Id  |          Operation                                             |         Name          |
-----------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                                       |                            |
|   1 |  SORT UNIQUE NOSORT                                 |                            |
|*  2 |   TABLE ACCESS BY GLOBAL INDEX ROWID   | EF_ACTL_EXPNS |
|*  3 |    INDEX RANGE SCAN                                   | EF_AEXP_PK       |
--------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   2 - filter("ACTL_EXPNS_AMT">0)
   3 - access("SRC_SYS"='CDW')

Again, note the NOSORT qualifier.

This is an extraordinary tuning technique in OLTP systems like SQL*Forms that return one page of detail at a time to the screen. A SQL with a DISTINCT, GROUP BY, or ORDER BY that uses an index to sort can return just the first page of matching rows without having to fetch the entire result set for a sort. This can be the difference between sub-second response time and several minutes or hours.

Full table Scans are not bad : 
Up to now, we've seen how indexes can be good. It's not always the case; sometimes indexes are no help at all, or worse: they make a query slower.

A b-Tree index will be no help at all in a reduced scan unless the WHERE clause compares indexed columns using >, <, LIKE, IN, or BETWEEN operators. A b-Tree index cannot be used to scan for any NOT style operators: eg. !=, NOT IN, NOT LIKE. There are lots of conditions, caveats, and complexities regarding joins, sub-queries, OR predicates, functions (inc. arithmetic and concatenation), and casting that are outside the scope of this article. Consult a good SQL tuning manual.

Much more interesting - and important - are the cases where an index makes a SQL slower. These are particularly common in batch systems that process large quantities of data.

To explain the problem, we need a new metaphor. Imagine a large deciduous tree in our front yard. It's Autumn, and it's our job to pick up all of the leaves on the lawn. Clearly, the fastest way to do this (without a rake, or a leaf-vac...) would be get down on hands and knees with a bag and work our way back and forth over the lawn, stuffing leaves in the bag as we go. This is a Full Table Scan, selecting rows in no particular order, except that they are nearest to hand. This metaphor works on a couple of levels: we would grab leaves in handfuls, not one by one. A Full Table Scan does the same thing: when a bock is read from disk, Oracle caches the next few blocks with the expectation that it will be asked for them very soon. Type this in SQL*Plus:

SQL> show parameter  db_file_multiblock_read_count 

Just to shake things up a bit (and to feed an undiagnosed obsessive compulsive disorder), we decide to pick up the leaves in order of size. In support of this endeavour, we take a digital photograph of the lawn, write an image analysis program to identify and measure every leaf, then load the results into a Virtual Reality headset that will highlight the smallest leaf left on the lawn. Ingenious, yes; but this is clearly going to take a lot longer than a full table scan because we cover much more distance walking from leaf to leaf.

So obviously Full Table Scan is the faster way to pick up every leaf. But just as obvious is that the index (virtual reality headset) is the faster way to pick up just the smallest leaf, or even the 100 smallest leaves. As the number rises, we approach a break-even point; a number beyond which it is faster to just full table scan. This number varies depending on the table, the index, the database settings, the hardware, and the load on the server; generally it is somewhere between 1% and 10% of the table.

The main reasons for this are :

  • As implied above, reading a table in indexed order means more movement for the disk head.
  • Oracle cannot read single rows. To read a row via an index, the entire block must be read with all but one row discarded. So an index scan of 100 rows would read 100 blocks, but a FTS might read 100 rows in a single block.
  • The db_file_multiblock_read_count setting described earlier means FTS requires fewer visits to the physical disk.
  • Even if none of these things was true, accessing the entire index and the entire table is still more IO than just accessing the table.

So what's the lesson here? Know our data! If our query needs 50% of the rows in the table to resolve our query, an index scan just won't help. Not only should we not bother creating or investigating the existence of an index, we should check to make sure Oracle is not already using an index. There are a number of ways to influence index usage; once again, consult a tuning manual. The exception to this rule - there's always one - is when all of the columns referenced in the SQL are contained in the index. If Oracle does not have to access the table then there is no break-even point; it is generally quicker to scan the index even for 100% of the rows.

Summary : 
Indexes are not a dark-art; they work in an entirely predictable and even intuitive way. Understanding how they work moves Performance Tuning from the realm of guesswork to that of science; so embrace the technology and read the manual.

                    http://www.orafaq.com/node/1403


Enjoy    :-)