Friday, May 28, 2010

Queries Running Very Slow

Query Running very slow:

This is the one which we often get, and the immediate answer would be, just go and take plan of the query and create some indexes if necessary. Lot of times this will not help you. There are so many possible reasons that we need to look.
If the query is running very slow collect some basic information before doing anything, this will be the first step to resolve this issue.

Information needs to collect when query is running slow

1. First execute the query and check how long it took? Is this query running very slow since it developed or it happened today? I mean yesterday or sometime before it was running fine but today it is very slow. The first thing you have to do is check the network traffic/Network errors for something when wrong in the network side, the best way to check is access some share drive which is located somewhere in network or use tnsping, traceroute, try pinging or ask network admin to check the packet size. If you notice any slow access then contacts network support. Sometimes this has been a problem. Which is out of scope of oracle database but created problem to the database, recently I faced the same kind of issue.

2. Network Issue:
Network is good no problems noticed then next step needs to check for any massive data loads done during last past night, are there any code changes, when was the last time the tables got analyzed, check the tables in the queries for proper indexes creation, check for tables, indexes and other objects for recent statistics, avoid using database functions, avoid using full table scan as much as possible, create partitions if necessary for faster access , use merge statements for huge updates and deletes, and so on. Check whether database statistics ran after database load. Simple way to run statistics is using DBMS_STAT.GATHER_TABLE_STATS

3. Statistics are fine. Still the query is slow. Next step needs to check is whether any database upgrades are done recently or any patches applied to higher version. Same queries with same data will run very slow in newer versions when upgrade from lower version. Then you might run with database CBO Bug. Contact Oracle support for more details regarding this bug.

4. I/O Issue:
Sometimes queries will be very slow due to more I/O from database to client.
Mechanism to avoid disk I/O and make the physical reads and writes more efficient
First get the largest amount of physical reads and logical reads and the simple way to get them are:

For Physical Reads:
Select disk_reads, sql_text from v$sqlarea where disk_reads > 10000 order by disk_reads desc;
For Logical Reads:
Select Buffer_gets, sql_text from v$sqlarea where buffer_gets > 200000 order by
Buffer_gets desc;
Multiple approaches must be made to cover all types of queries

5. Memory Issue:
Pin (Cache) most used PL/SQL statement into memory
Syntax: dbms_shared_Pool.keep(Object_name)
Few queries which would help you to resolve the memory related issues are:
Select name, value from gv$parameter where name in (‘db_block_buffers’, ‘db_block_size’, ‘shared_pool_size’, ‘sort_area_size’);
Check the following:
select 1-(sum(decode(name, 'physical reads', value,0))/ (sum(decode(name, 'db block gets', value,0)) + (sum(decode(name, 'consistent gets', value,0))))) * 100 "Read Hit Ratio" from gv$sysstat;

Read Hit Ratio
98.99
Hit ratio below 90-95% is signed as poor indexing;
Check the Shared Pool Size:
It is important to look at the shared_pool_size for proper sizing. With a greater amount of procedures, packages and triggers being utilized with Oracle, the shared_pool_size makes up a much greater portion of the Oracle SGA. This is the memory allocated for the library and data dictionary cache. If the shared_pool_size is set too low then you will not get the full advantage of your DB_BLOCK_BUFFERS.

How to find the data dictionary cache miss ratio?
Select sum(gets) “Gets”, sum(getmisses) “Misses”, (1-(sum(getmisses)/(sum(gets)+ sum(getmisses)))) * 100 “HitRate”
From gv$rowcache;

Gets          Misses           HitRate
11000           558            96.5

This is a good ratio does not require any further action.
How to find library Cache Hit Ratio?
select sum(pins) Executions, sum(pinhits) "Execution Hits", ((sum(pinhits)/sum(pins)) * 100)phitrat, sum(reloads)Misses, ((Sum(pins)/(sum(pins) + sum(reloads))) * 100) hitrat from gv$librarycache;

EXECUTIONS Execution Hits PHITRAT MISSES HITRAT
---------- -------------- ---------- ---------- ----------
1126456014 1121589121 99.5679465 102177 99.9909302

How much memory is left for shared_pool_size?
select to_number(gv$parameter.value) value, gv$sgastat.bytes, (gv$sgastat.bytes/gv$parameter.value)*100 “Percent Free
from gv$sgastat, gv$parameter where gv$sgastat.name = 'free memory and gv$ parameter .name = ‘shared_pool_size;

Shared Pool Size Free Bytes Percent Free

100,000,000 82,278,960 82.278


Regards,
Sowmya (OSR)

Wednesday, May 12, 2010

ORA-0600: Internal Code Error

 ORA-0600: Internal Code Error


Explanation: Any ORA-600 error indicates Oracle has detected an internal inconsistency or a problem which it doesn’t know how best to address. These are not necessarily bugs and can occur for reasons such as Operating System, IO Problems, and Running out of some resources and so on.

Cause: This generic internal error number for oracle program exceptions. This indicates that a process has encountered an exceptional condition.

Note: This type of errors is raised from kernel code of the Oracle RDBMS software when there is an inconsistency in detected or unexpected condition is noticed. This situation need not be considered as bug all the time; it might be caused by problems with operating system, lack of resources etc.

With the ORA-600 error comes with a list of arguments in square brackets. The first of these arguments tells us from where in the code the error was caught and this is the key information for identifying the problem. This can be either numbers or arguments.

Whenever there is an ORA-600 error is raised a trace file is generated in either Udump or Bdump depending on whether the error was caught in a user or a background process. The information is also present in the alert log.

Remedy: Contact Oracle Support.

• Report as bug if the first argument is the internal error number

The most useful items are:

• Check the alert log for the instance or any other instances in a parallel server environment. Alert log is present in Background_dump_dest

• Check the trace file for the failing session and other related trace files

• Check whether anything got changed recently in the system environment such OS been upgraded, patches, database files got restored etc.

• Make sure whether the problem is reproducible or one time occurrence

The following is partial list of the ORA-00600 known bugs and issues. You can find complete list and details about these bugs at Oracle Metalink.

• Partial list of ORA-00600 known bugs and issues:

1. An Oracle ORA-00600 [729]: indicates a memory leak in the UGA. Setting diagnostic events 10262 ignores this error, that is, no ORA-00600 is produced even if a memory leak is detected.

2. ORA-00600 [16365]: indicates a Half Duplex Violation. Shared servers and dispatchers cannot both be writing to the same virtual circuit at the same time because they communicate with a half-duplex protocol (TTC protocol).

3. ORA-00600 [kghpih:ds] : and ORA-00600 [17175] may occur in oracle 9.2.0.6 when two processes work on the same library cache object. Fixed in oracle 9.2.0.7... Note: 300604.1

4. An Oracle ORA-00600 [3020]: Bug.2322620 - RECOVERY HALTED ON PHYSICAL STANDBY WITH ORA-600 AND ORA-10564 Details: ORA-600 [3020] can occur on recovery of LOB data. Fixed in 8.1.7.5 Patch available. This error occurred on our database recovery on the primary server.

5. ora-00600 [12333]: Most probably a network error that may have caused bad data to be received or the client application may have sent the wrong data in the network buffer.

6. Oracle ORA-00600 [12700]: Indicates corruption in index, table or the mapping between them. Upon this discovery, oracle marks the whole block as corrupt and users will not have normal access to the data from the corrupted block. This can also break referential integrity and should be referred to oracle.

7. ORA-00600 [lxhcsn]: Shutdown may dump ORA-7445. Known Bug 2311529

Conclusion: This must be taken as a serious issue and contact Oracle support or raise a SR for unknown issues

Regards,
Sowmya (OSR)

Common Oracle Errors


Wednesday, April 28, 2010

Oracle Grid- Creating User Accounts

Hello,

I am going to discuss more about Oracle Grid - creating administrator user, super administrator user, creating targets, generating reports, scheduling jobs, setting alerts,setting email notification and so on.
In this post i am going to create two administrator user accounts (warning_admin, critical_admin) and super administrator user account (super_admin) using Oracle Grid.
Administrator User Accounts: Adminsitrator are deatabase users defined in the management repository who can log into Enterprise Manager to perform management tasks.
Super Administrator User Account:The Super Administrator can choose to grant certain privileges to certain administrators to access and manage certain targets based on the requirement. Thus, the superadministrator can divide the work intelligently among the adminsitrators.

Creating Super Administrator User Super_Admin using Oracle Grid
Super_Admin
• Go to Setup -> click on Administrators
• Click Createlike -> provide the following information
• Name: Super_Admin
• Password:
• Confirm password:
• Check on Super Administrator
• Click Finish

Creating Adminsitrator User Accounts Warning_Admin and Critical_Admin

Warning_Admin
Login to Grid user Super Administrator user account Super_Admin
Create 2 User Accounts
Note: This 2 users are created just to differentiate critical alerts and warning alerts and set email notifications to groups based on the priority. It is sometimes needed to avoid your boss getting all warning emails, so make sure that you have 2 email groups with your boss included in criticals alerts and not in warning alerts.
Warning_Admin: This account is created to set for all the alerts which come under warning messages. A warning email is sent to everyone in the team except your Boss.
• Go to setup -> Adminitrators
• Click on Createlike -> Provide following information
• Name: warning_admin
• Password:
• Confirm Password:
• Email:
• Don’t check on super administrator
• Click next Roles Page
• Make the role as public
• Click next to system privileges
• Check on view any target
• Click next to Target
• Browse Type-> All target type and move all to selected targets
• Click next to target privileges
• Click next to Review and click Finish

Critical_Admin
Login into Oracle Grid using Super_Admin
Create administrator user Critical_Admin
Critical_Admin: This account is created to set for all the alerts which come under critical messages. A Critical email is sent to everyone in the team including your Boss. The critical email is .......
• Go to setup -> Administrators
• Click on createlike -> Provide following information
• Name: critical_admin
• Password:
• Confirm Password:
• Email:
• Don’t check on super administrator
• Click next roles Page
• Make the role as Public
• Click Next to system privileges
• Check on view any target
• Click next to Target
• Browse Type -> All target type and move all to selected targets
• Click next to target privileges
• Click next to Review and click finish

Regards,
Sowmya (OSR)

Tuesday, April 20, 2010

RMAN- Block Change Tracking

Hello,
Block Change Tracking
As data blocks changes, the background process change tracking writer (CTWR) tracks all the changes made to the data block in a private area of memory. When we issue commit on these changes, the block change tracking information is copied to a shared area in large pool called the CTWR buffer. CTWR process writes the information from the CTWR RAM buffer to the change tracking file. Without BCT enabled or in cases when change tracking information cannot be used, RMAN has to read every block in the datafile during incremental backup. Each block contains last system change number (SCN) of its last modification. The block is copied only if its SCN is higher or equal to the base backup SCN. To be precise, the backup SCN is a checkpoint SCN that was made right before the backup. Since, usually, only handful of blocks is changed between incremental backups, RMAN does a lot of useless work reading the blocks not required for backup.
Block change tracking provides a way to identify the blocks required for backup without scanning the whole datafile. After that RMAN need only read blocks that are really required for this incremental backup.

How to Enable/Disable Block change tracking?
Enable
Syntax: SQL>Alter database enable block change tracking (syntax when OMF is enabled)
Or
Alter database enable block change tracking using file os_file_name;
Using os_file_name syntax allows you to define the location of the change tracking file on the OS or you can omit this using OMF (oracle Managed files)
By default the change tracking file is created as an Oracle Managed File (OMF) in the location pointed to by the DB_CREATE_FILE_DEST parameter.
Disable
SQL>alter database disable block change tracking;

How to monitor the status of block change tracking?
SQL>select filename, status, bytes from v$block_change_tracking

How can we view the size of CTWR dba buffer?
Select * from v$sgastat where name like ‘CTWR%’;

Regards,
Sowmya (OSR)

Monday, April 19, 2010

RMAN-Incremental Backup

Hello,
Incremental Backups using RMAN:

Incremental Backups: RMAN incremental backups back up only data file block that have been changed since last backup. One can make incremental backups at Database level, Tablespace level or datafile level.
The main theme of incremental backup is to back up only those data blocks that have been changed since previous backup.
The reasons for going incremental backup as a part of our backup strategy are:
1. To reduce the amount of time needed for taking backups.
2. To save network bandwidth when backing up over network.
3. For better backup performance.
4. To reduce backup size for Non-Archive log databases, instead of going with full database backup one can go with incremental backups.

Incremental Backup Algorithm:
Data is stored in the data blocks. Each data block in a data file contains system change number SCN, which is the SCN at which most recent changes are made to the block. During incremental backup RMAN reads the SCN of each data block in the input file and compares this with the checkpoint SCN of the parent incremental backup. If the SCN in the input data block Is greater than or equal to the checkpoint SCN of the parent, then RMAN copies the block.

Level 0 and Level 1 Backups:
Level 0: Level 0 is just like a full backup which is the base of the subsequent incremental backups, copies all blocks containing data, backing the data file up into a backup set.
The main difference between the full backup and incremental level 0 backups is , full backup is never included in the incremental backup strategy.
Level 1: With reference to the previous level 0 backup or level 1 backup it backs up the modified data blocks.

Types of Level 1 incremental backups:
Differential Backup: Backus up all blocks changed after the most recent incremental backup at level 1 or 0
Cumulative backup: Backups up all blocks changed after the most recent incremental backup at level 0
By default incremental backups are differential.
RMAN> Backup Incremental level 1 Database; (example of differential backup)
RMAN> Backup Incremental level 1 cumulative database; (blocks changed since level 0- Cumulative Incremental Backups.

Incremental Backup Strategy:
Choose a backup schema according to an acceptable MTTR (mean time to recover). For example, you can implement a three –level backup schema so that a full or level 0 backup is taken monthly, a cumulative level 1 is taken weekly, and a differential level 1 is taken daily.

How often we can take full or incremental backups?
When deciding how often we can take full or incremental backups, a good rule of thumb is to take a new level 0 backup whenever 50% or more of the data has changed. If the rate of change to your database is predictable, then you can observe the size of your incremental backups to determine new level 0 is appropriate.
The following query will help you to determine the rate of changed blocks.

SELECT FILE#, INCREMENTAL_LEVEL, COMPLETION_TIME, BLOCKS, DATAFILE_BLOCKS
FROM V$BACKUP_DATAFILE
WHERE INCREMENTAL_LEVEL > 0
AND BLOCKS / DATAFILE_BLOCKS > .5
ORDER BY COMPLETION_TIME;

Making Incremental Backups : Backup Incremental
After starting RMAN, run the BACKUP INCREMENTAL command at the RMAN prompt. This example makes a level 0 incremental backup of the database:
BACKUP INCREMENTAL LEVEL 0 DATABASE;
This example makes a differential level 1 backup of the SYSTEM tablespace and datafile tools01.dbf. It will only back up those data blocks changed since the most recent level 1 or level 0 backup:
BACKUP INCREMENTAL LEVEL 1
TABLESPACE SYSTEM
DATAFILE 'ora_home/oradata/trgt/tools01.dbf';
This example makes a cumulative level 1 backup of the tablespace users, backing up all blocks changed since the most recent level 0 backup.
BACKUP INCREMENTAL LEVEL = 1 CUMULATIVE
TABLESPACE users;

Regards,
Sowmya (OSR)
http://usdba.blogspot.com/2010/04/rman-incremental-backup.html

Thursday, April 1, 2010

RMAN -Practicals

Hello,
Connecting to RMAN
1.Open the DOS command prompt.
2.C:>rman
3.RMAN>
4.RMAN>exit
Connecting to the target database
1.Open the DOS command prompt.
2.C:>SET ORACLE_SID=kendb
3.RMAN> connect target
Or
rman target / NOCATALOG
Or
rman target/
4.RMAN>exit
Creating the Repository in the Recovery Catalog Database
RMAN repository in a separate database called RCMDB.
Add entry in tnsnames.ora file.
Make sure RCMDB is in ArchiveLog Mode.
Make sure Target Database (KENDB) is in archivelog mode.
1. sqlplus to RCMDB as SYSDBA
Set oracle_sid=RCMDB
Sqlplus “/as sysdba”
Sql>select name from v$database; (Verify you connected to rcmdb)
2. CREATE USER RMANUSER
IDENTIFIED BY RMANUSER
DEFAULT TABLESPACE USERS
TEMPORARY TABLESPACE TEMP;
3. sql>grant connect, resource,recovery_catalog_owner to rmanuser;
4.sql>exit;
5. Invoke RMAN executable
RMAN
RMAN>connect catalog rmanuser/rmanuser@RCMDB
6. Create catalog for RMANUSER
RMAN>CREATE CATALOG;
7. RMAN>EXIT;
Connecting to Target and Recovery Catalog Database
1. C:>set oracle_sid=TESTDB
2. C:>RMAN
3. RMAN>CONNECT TARGET
4. RMAN>CONNECT CATALOG RMANUSER/RMANUSER@RCMDB
5. RMAN>EXIT;
Or
Instead or step3,4 and 5 You can use following:
C:>RMAN TARGET / CATALOG RMANUSER/RMANUSER@RCMDB
Registering the Target Database.
1. C:>SET ORACLE_SID=TESTDB
2. C:>RMAN
3. RMAN>CONNECT TARGET
4. RMAN>CONNECT CATALOG RMANUSER/RMANUSER@RCMDB
5. RMAN>REGISTER DATABASE;
6. RMAN>EXIT;
Setting the Retention Policy
1. Launch RMAN again
2. C:>rman target/
3. RMAN>configure retention policy to redundancy 2;
4. RMAN>show retention policy;
5. RMAN>configure retention policy to recovery window of 3 days;
6. RMAN>configure retention policy clear;
7. RMAN>exit;
Configuring the Default Device Type
1. C>rman target/
2. RMAN>configure default device type to sbt; (for tape)
3. RMAN>configure default device type to disk;
4. RMAN>show default device type;
5. RMAN>configure default device type clear;
6. RMAN>exit;
format for device type disk
Configuring Controlfile Autobackup
1. C>rman target/
2. RMAN>configure controlfile autobackup on;
3. RMAN> configure controlfile autobackup format for device type disk to 'f:\backup\kendb\rman\%F';
4. RMAN>show controlfile autobackup;
5. RMAN>show controlfile autobackup format;
6. RMAN>configure controlfile autobackup clear;
7. RMAN>configure controlfile autobackup format for device type disk clear;
8. RMAN>exit;
Configuring the Default Backup Type
1. C>rman target/
2. RMAN>configure device type disk backup type to backupset;
3. RMAN>configure device type disk backup type to copy;
4. RMAN>configure device type disk backup type to compressed backupset;
5. RMAN>show device type;
6. RMAN>backup database;
7. RMAN>list backup;
8. RMAN>configure device type disk clear;
9. RMAN>exit;
Parallelizing backups
1. C>set oracle_sid=kendb
2. C>rman target/
3. RMAN>configure device type disk parallelism 4;
4. RMAN>backup datafile 2, 3;
RMAN>exit
Backup entire Database
1. C>set oracle_sid=kendb
2. C>rman
3. RMAN>CONNECT TARGET
4. RMAN>CONNECT CATALOG RMANUSER/RMANUSER@RCMDB;
5. RMAN>CONFIGURE DEVICE TYPE DISK CLEAR;
6. RMAN>BACKUP DATABASE;
OR
7. RMAN> BACKUP DATABASE FORMAT ‘O:\ORABACKUP\KENDB\BACKUP.RMAN\db_%u_%d_%s_%p’;
OR
8. RMAN>BACKUP DATABASE MAXSETSIZE=500M;
9. RMAN>EXIT;
Backup Tablespace
1. C>set oracle_sid=kendb
2. C>rman target/
3. RMAN>backup tablespace system;
4. RMAN>backup tablespace system, sysaux;
Backup Datafile
1. C>set oracle_sid=kendb
2. C>rman target/
3. RMAN>report schema;
4. RMAN>backup datafile 3;
5. RMAN>backup datafile 2,4;
6. RMAN>exit;
Backup Controlfile
1. C>rman target/
2. RMAN>backup current controlfile;
3. RMAN>exit;
Creating Compressed Backups
1. C>set oracle_sid=kendb
2. C>rman target/
3. RMAN>backup as compressed backupset datafile 3;
RMAN>exit;
Backup ArchiveLogs
1. C>set oracle_sid=kendb
2. C>rman
3. RMAN>backup archivelog all;
OR
4. RMAN>backup archivelog all delete input;
(This deletes archivelog files after they have been backed up.)
5. Backup Archivelogs generated in the past 2 days.
6. RMAN>backup archivelog from time ‘sysdate-2’ until time ‘sysdate’;
7. RMAN>exit;
Performing Incremental Backups
1. C>set oracle_sid=kendb
2. C>rman target/
3. RMAN>backup incremental level 0 database;
Incremental backups may be taken at Tablespace or datafile level.
OR
4. RMAN>BACKUP INCREMENTAL LEVEL 0 DATAFILE 2, 4;
5. RMAN>BACKUP INCREMENTAL LEVEL 0 TABLESPACE USERS;
Create Incremental Level 1 backups
6. RMAN>backup incremental level 1 tablespace users;
7. RMAN>backup incremental level 1 database;
Create Cumulative Level 1 backups
8. RMAN>backup incremental level 1 cumulative tablespace users;
9. RMAN>backup incremental level 1 cumulative database;
10. RMAN>backup incremental level 1 cumulative datafile 2, 4;
11. RMAN>exit;
Create Image Copies
1. C>set oracle_sid=kendb
2. C>rman target/
3. RMAN>backup as copy database;
OR
4. RMAN>backup as copy tag “users_wkly_bkp” tablespace users;
OR
5. RMAN>backup as copy datafile 4;
6. RMAN>exit;
LIST COMMAND
Which backups are currently available.
1. C>set oracle_sid=kendb
2. C>rman target/
3. rman>list backup by backup;
4. rman>list backupset by backup;
5. rman>list backup by backup verbose;
6. rman>list backup by backup summary;
7. rman>list backup by file;
8. rman>list copy of database; image copies
9. rman>list copy of tablespace users;
10. rman>list copy of datafile 3;
11. rman>list incarnation; to view different incarnation
Identify files that are in need of backup
1. c>set oracle_sid=kendb
2. c>rman target/
3. rman>configure retention policy to redundancy 2;
4. rman> report need backup;
5. rman> report need backup days=7 database; files that need a minimum of 7 days of logfiles to be applied in the event of recovery.
6. rman>exit;
Report Obsolete
1. c>set oracle_sid=kendb
2. c> rman target/
3. rman> report obsolete;
4. rman> delete obsolete;
5. rman>exit;

Regards,
Sowmya (OSR)