Saturday, September 8, 2012

How to recover from a loss of one redo log member of a multiplexed group

This scenario describes what happens when you lose one redo log member of a multiplexed group.
It could happen for a simply media failure or because some one "needed" to free some space and saw that .log file!!!
It happens to me years ago and that was told to me by a system administrator.

Anyway when you have multiplexed your redo log group using several members as Oracle best practices suggest and your disks begin to experience problems your database will continue to function since it's still able to write to one redo log member.
If your disk is completely out you can opt to some options whether or not you have a new disk!
With a new disk you can drop and create again the "lost" redo log member to the original location, otherwise you have to perform the same two steps (drop and create) choosing a different location.
Even if you have already lost your redo log member Oracle will arise the ORA-01609 error when you try to drop a member of a CURRENT group. Querying the status column of V$LOG view you can have several results and generally they are:
CURRENT when that log file is "currently" written by the log writer process (ps -ef|grep lgwr),
ACTIVE when that log file is required in case of a crash recovery (even if it's already archived)
INACTIVE when that log file is not required in case of a crash recovery.

So when you try to drop your "failed" redo log member and you receive the ORA-01609 error just force the database to switch to the next redo log group. 

Let's look it with an example. Here is my redo log configuration:
SQL> select member, a.group#, a.status, b.status from v$log a, v$logfile b where  a.group# = b.group#
  2  order by a.group#, member;

MEMBER                                                           GROUP# STATUS           STATUS
------------------------------------------------------------ ---------- ---------------- -------
/home/oracle/app/oracle/oradata/orcl/redo01.log                       1 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo01b.log                      1 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo02.log                       2 CURRENT
/home/oracle/app/oracle/oradata/orcl/redo02b.log                      2 CURRENT
/home/oracle/app/oracle/oradata/orcl/redo03.log                       3 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo03b.log                      3 INACTIVE
As you can see I have three redo log groups formed by two multiplexed members and currently the redo log group number 2 is the CURRENT one. Now I simulate the lost of the redo log member named 'redo02b.log'.
[oracle@localhost ~]$ cd /home/oracle/app/oracle/oradata/orcl/
[oracle@localhost orcl]$ rm redo02b.log
[oracle@localhost orcl]$ ll redo*
-rw-rw---- 1 oracle oracle 52429312 Sep  5 12:58 redo01b.log
-rw-rw---- 1 oracle oracle 52429312 Sep  5 12:58 redo01.log
-rw-rw---- 1 oracle oracle 52429312 Sep  5 22:30 redo02.log
-rw-rw---- 1 oracle oracle 52429312 Sep  5 12:55 redo03b.log
-rw-rw---- 1 oracle oracle 52429312 Sep  5 12:55 redo03.log
As you cann see from the alert log appears the error
...
Errors in file /home/oracle/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_lgwr_8172.trc:
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00312: online log 2 thread 1: '/home/oracle/app/oracle/oradata/orcl/redo02b.log'
ORA-27037: unable to obtain file status
Linux Error: 2: No such file or directory
...
Now have a look at how the status column of V$LOGFILE view change (from NULL to INVALID) for the lost member
SQL> select member, a.group#, a.status, b.status from v$log a, v$logfile b where  a.group# = b.group#
  2  order by a.group#, member;

MEMBER                                                           GROUP# STATUS           STATUS
------------------------------------------------------------ ---------- ---------------- -------
/home/oracle/app/oracle/oradata/orcl/redo01.log                       1 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo01b.log                      1 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo02.log                       2 CURRENT
/home/oracle/app/oracle/oradata/orcl/redo02b.log                      2 CURRENT          INVALID
/home/oracle/app/oracle/oradata/orcl/redo03.log                       3 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo03b.log                      3 INACTIVE
I've lost a member of the CURRENT redo log group so I cannot drop it
SQL> alter database drop logfile member '/home/oracle/app/oracle/oradata/orcl/redo02b.log';
alter database drop logfile member '/home/oracle/app/oracle/oradata/orcl/redo02b.log'
*
ERROR at line 1:
ORA-01609: log 2 is the current log for thread 1 - cannot drop members
ORA-00312: online log 2 thread 1: '/home/oracle/app/oracle/oradata/orcl/redo02.log'
ORA-00312: online log 2 thread 1: '/home/oracle/app/oracle/oradata/orcl/redo02b.log'
I have to force a redo log switch:
SQL> alter system switch logfile;

System altered.
Now my redo log group state is as the following:
SQL> select member, a.group#, a.status, b.status from v$log a, v$logfile b where  a.group# = b.group#
  2  order by a.group#, member;

MEMBER                                                           GROUP# STATUS           STATUS
------------------------------------------------------------ ---------- ---------------- -------
/home/oracle/app/oracle/oradata/orcl/redo01.log                       1 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo01b.log                      1 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo02.log                       2 ACTIVE
/home/oracle/app/oracle/oradata/orcl/redo02b.log                      2 ACTIVE           INVALID
/home/oracle/app/oracle/oradata/orcl/redo03.log                       3 CURRENT
/home/oracle/app/oracle/oradata/orcl/redo03b.log                      3 CURRENT
And I can successfully drop and recreate my lost redo log member:
SQL> alter database drop logfile member '/home/oracle/app/oracle/oradata/orcl/redo02b.log';

Database altered.

SQL> select member, a.group#, a.status, b.status from v$log a, v$logfile b where  a.group# = b.group#
  2  order by a.group#, member;

MEMBER                                                           GROUP# STATUS           STATUS
------------------------------------------------------------ ---------- ---------------- -------
/home/oracle/app/oracle/oradata/orcl/redo01.log                       1 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo01b.log                      1 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo02.log                       2 ACTIVE
/home/oracle/app/oracle/oradata/orcl/redo03.log                       3 CURRENT
/home/oracle/app/oracle/oradata/orcl/redo03b.log                      3 CURRENT

SQL> alter database add logfile member  '/home/oracle/app/oracle/oradata/orcl/redo02b.log' to group 2;

Database altered.

SQL> select member, a.group#, a.status, b.status from v$log a, v$logfile b where  a.group# = b.group#
  2  order by a.group#, member;

MEMBER                                                           GROUP# STATUS           STATUS
------------------------------------------------------------ ---------- ---------------- -------
/home/oracle/app/oracle/oradata/orcl/redo01.log                       1 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo01b.log                      1 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo02.log                       2 ACTIVE
/home/oracle/app/oracle/oradata/orcl/redo02b.log                      2 ACTIVE           INVALID
/home/oracle/app/oracle/oradata/orcl/redo03.log                       3 CURRENT
/home/oracle/app/oracle/oradata/orcl/redo03b.log                      3 CURRENT

What's happening ? My new redo log member is still INVALID ? What does it mean ?
If you look at the Oracle documentation INVALID could mean: inaccessible (as it was before) or just created (as I did few lines above).
Let's look what it happens when I force logfiles to switch:

SQL> alter system switch logfile;

System altered.

SQL> select member, a.group#, a.status, b.status from v$log a, v$logfile b where  a.group# = b.group#
  2  order by a.group#, member;

MEMBER                                                           GROUP# STATUS           STATUS
------------------------------------------------------------ ---------- ---------------- -------
/home/oracle/app/oracle/oradata/orcl/redo01.log                       1 CURRENT
/home/oracle/app/oracle/oradata/orcl/redo01b.log                      1 CURRENT
/home/oracle/app/oracle/oradata/orcl/redo02.log                       2 INACTIVE
/home/oracle/app/oracle/oradata/orcl/redo02b.log                      2 INACTIVE         INVALID
/home/oracle/app/oracle/oradata/orcl/redo03.log                       3 ACTIVE
/home/oracle/app/oracle/oradata/orcl/redo03b.log                      3 ACTIVE

6 rows selected.

SQL> alter system switch logfile;

System altered.

SQL> select member, a.group#, a.status, b.status from v$log a, v$logfile b where  a.group# = b.group#
  2  order by a.group#, member;

MEMBER                                                           GROUP# STATUS           STATUS
------------------------------------------------------------ ---------- ---------------- -------
/home/oracle/app/oracle/oradata/orcl/redo01.log                       1 ACTIVE
/home/oracle/app/oracle/oradata/orcl/redo01b.log                      1 ACTIVE
/home/oracle/app/oracle/oradata/orcl/redo02.log                       2 CURRENT
/home/oracle/app/oracle/oradata/orcl/redo02b.log                      2 CURRENT
/home/oracle/app/oracle/oradata/orcl/redo03.log                       3 ACTIVE
/home/oracle/app/oracle/oradata/orcl/redo03b.log                      3 ACTIVE

6 rows selected.
As you can see the database marked the 'redo02b.log' as NULL and this means available to be used. That's all.

214 comments:

«Oldest   ‹Older   201 – 214 of 214
Yoona said...

What wonderful creativity – a joy to view! keep up the nice quality writing, it is rare to see a great blog like this one today.
카지노사이트
https://yhn876.com 카지노사이트

joyce said...

Incredible! You got such a nice website! Keep it up! Check out this website too if you have time
카지노사이트
https://yhn876.com/ 카지노사이트

Trini said...

Amazing blog. This is really helpful and informative. Please come and visit our website. Thank you! Have a great day ahead.
카지노사이트
https://yhn777.com 카지노사이트

Yoona said...

I am very happy! I came across this during my search for something regarding this. Great work! Keep it up! Please do visit our website below.
카지노사이트
https://yhn876.com 카지노사이트

ANNALIZA said...

HEY! THIS KIND OF BLOG IS VERY WORTH IT TO READ
TRY TO SPEND YOUR TIME READING THIS.
JUST CLICK OUR LINK BELOW!
카지노사이트
https://yhn777.com 카지노사이트

sabrina said...

Wonderful blog! This is definitely worth-reading. Thank you for this informative content. Continue creating blogs like this. Visit our website too for more creative contents.
카지노사이트
https://yhn777.com 카지노사이트

카지노사이트 said...

Both Android and iOS devices let you limit when an app can access your location. Always limit location access to while you're using the app or make the app ask each time, and never let an app continuously track your location in the background without an exceptionally good reason. Reset your advertising ID in your smartphone's settings.
https://www.yert200.com

카지노사이트 said...

If you're worried about someone knowing your browsing history, use a VPN, but be careful about which ones. (Cyphers says a VPN won't protect your location information as well.)
https://www.shine900.com

카지노사이트 said...

"Time and time again, whenever the burden is on users to opt out of something, the vast amount of users are not going to opt out of that thing," said Cyphers.
https://www.gain777.com

카지노사이트 said...

The selection of Buckley to serve in that capacity, however, could come back to haunt the Democrats on the committee who selected him. Yahoo News has obtained a previously unpublished 2019 report compiled by the Department of Homeland Security’s watchdog office showing that investigators urged the CIA to take action against Buckley for his alleged retaliation against a whistleblower, a conclusion that would likely be troubling to potential witnesses who might testify in the Jan. 6 inquiry.
https://www.kkr789.com

카지노사이트 said...

The authors of the report recommended that “at minimum” the CIA determine “whether [their] findings affect the security clearances” of Buckley and several fellow senior officials — a serious rebuke that would have affected his future government contract work as well as his tenure with the highly sensitive Capitol riot investigation.
https://www.aace777.com

카지노사이트 said...

The target of Buckley’s retaliation, former CIA IG official Andrew Bakaj, has yet to receive any remedy for a series of adverse actions that affected his career, including being put on administrative leave and having his security clearance suspended after cooperating with an external investigation into potential evidence manipulation at the CIA inspector general’s office.
https://www.qqueen700.com

카지노사이트 said...

Bakaj’s superiors became frustrated, however, when he did not inform them he had responded to requests for information from the intelligence community inspector general, who was tasked with independent reviews affecting agencies within the intelligence community.
https://www.rcasinosite.com

카지노사이트 said...

He waited over five years for the Department of Homeland Security inspector general to conclude an impartial review of his retaliation complaints, which vindicated him
https://www.hgame789.com

«Oldest ‹Older   201 – 214 of 214   Newer› Newest»