Wednesday, April 24, 2013

How to create an archival backup and maintain it forever

Sometimes it could happen to retain a backup for a very long time, even forever that is it should be maintained until you manually remove it.

This post is related on how to keep a backup forever.

In particular the syntax to have a backup retained forever is using the keep forever option: these kind of backups are also called archival backups, that is a self-contained backups, including all the necessary files (datafiles and archived redo logs) to recreate a consistent backup.

At the same time to successfully complete and maintain a backup forever fewer requisites are required, but let's begin issuing the backup command.
[oracle@vsi08 ~]$ rman target /

Recovery Manager: Release 11.2.0.1.0 - Production on Fri Apr 19 12:25:00 2013

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

connected to target database: PROD (DBID=223010867)

RMAN> backup database keep forever;

Starting backup at 19-04-2013 12:25:27
current log archived

using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=17 device type=DISK
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup command at 04/19/2013 12:25:28
RMAN-06522: KEEP FOREVER option is not supported without the recovery catalog
As you can deduct from the error message you cannot issue a keep forever option using the target database control file: when using keep forever option you must specify a recovery catalog schema on a dedicated database.
So I'm going to use my new recovery catalog schema on RCAT database.
[oracle@vsi08 ~]$ rman target / catalog rman/rman@RCAT

Recovery Manager: Release 11.2.0.1.0 - Production on Fri Apr 19 12:26:07 2013

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

connected to target database: PROD (DBID=223010867)
connected to recovery catalog database
I can issue the same backup command...
RMAN> backup database keep forever;

Starting backup at 19-04-2013 12:26:16
current log archived

allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=17 device type=DISK
backup will never be obsolete
archived logs required to recover from this backup will be backed up
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00001 name=/opt/app/oracle/oradata/PROD/system01.dbf
input datafile file number=00002 name=/opt/app/oracle/oradata/PROD/sysaux01.dbf
input datafile file number=00003 name=/opt/app/oracle/oradata/PROD/undotbs01.dbf
input datafile file number=00004 name=/opt/app/oracle/oradata/PROD/users01.dbf
channel ORA_DISK_1: starting piece 1 at 19-04-2013 12:26:18
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup command at 04/19/2013 12:26:18
ORA-19811: cannot have files in DB_RECOVERY_FILE_DEST with keep attributes
... but it fails once again.
The second restriction while using the keep forever option is related on the flash recovery area: you cannot create those archival backups into the FRA and if you try to do so, the backup fails and an error is thrown.
As you can see I'm currently using a flash recovery area.
SQL> show parameter DB_RECOVERY_FILE_DEST

NAME         TYPE  VALUE
------------------------------------ ----------- ------------------------------
db_recovery_file_dest       string  /opt/app/oracle/flash_recovery
       _area
db_recovery_file_dest_size      big integer 4977M
To complete my backup I need to use also the format option specifying a directory where creating the archival backups:
RMAN> backup database format '/home/oracle/backup/%U' keep forever;

Starting backup at 19-04-2013 12:35:12
current log archived

using channel ORA_DISK_1
backup will never be obsolete
archived logs required to recover from this backup will be backed up
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00001 name=/opt/app/oracle/oradata/PROD/system01.dbf
input datafile file number=00002 name=/opt/app/oracle/oradata/PROD/sysaux01.dbf
input datafile file number=00003 name=/opt/app/oracle/oradata/PROD/undotbs01.dbf
input datafile file number=00004 name=/opt/app/oracle/oradata/PROD/users01.dbf
channel ORA_DISK_1: starting piece 1 at 19-04-2013 12:35:14
channel ORA_DISK_1: finished piece 1 at 19-04-2013 12:35:59
piece handle=/home/oracle/backup/11o7fhc2_1_1 tag=TAG20130419T123513 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:45

using channel ORA_DISK_1
backup will never be obsolete
archived logs required to recover from this backup will be backed up
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
including current SPFILE in backup set
channel ORA_DISK_1: starting piece 1 at 19-04-2013 12:35:59
channel ORA_DISK_1: finished piece 1 at 19-04-2013 12:36:00
piece handle=/home/oracle/backup/12o7fhdf_1_1 tag=TAG20130419T123513 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01

current log archived
using channel ORA_DISK_1
backup will never be obsolete
archived logs required to recover from this backup will be backed up
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=94 RECID=47 STAMP=813155760
channel ORA_DISK_1: starting piece 1 at 19-04-2013 12:36:01
channel ORA_DISK_1: finished piece 1 at 19-04-2013 12:36:02
piece handle=/home/oracle/backup/13o7fhdh_1_1 tag=TAG20130419T123513 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01

using channel ORA_DISK_1
backup will never be obsolete
archived logs required to recover from this backup will be backed up
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
including current control file in backup set
channel ORA_DISK_1: starting piece 1 at 19-04-2013 12:36:03
channel ORA_DISK_1: finished piece 1 at 19-04-2013 12:36:05
piece handle=/home/oracle/backup/14o7fhdi_1_1 tag=TAG20130419T123513 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:02
Finished backup at 19-04-2013 12:36:05
Listing the archival backups you can notice a new line after the Piece Name: it says that backup is an archival backup created using the KEEP option and it will be maintained FOREVER. All those archival backups are AVAILABLE, that is RMAN is able to find them on the file system.
RMAN> list backup;

List of Backup Sets
===================

BS Key  Type LV Size       Device Type Elapsed Time Completion Time    
------- ---- -- ---------- ----------- ------------ -------------------
403     Full    835.42M    DISK        00:00:44     19-04-2013 12:35:58
        BP Key: 406   Status: AVAILABLE  Compressed: NO  Tag: TAG20130419T123513
        Piece Name: /home/oracle/backup/11o7fhc2_1_1
        Keep: BACKUP_LOGS        Until: FOREVER        
  List of Datafiles in backup set 403
  File LV Type Ckp SCN    Ckp Time            Name
  ---- -- ---- ---------- ------------------- ----
  1       Full 1410646    19-04-2013 12:35:14 /opt/app/oracle/oradata/PROD/system01.dbf
  2       Full 1410646    19-04-2013 12:35:14 /opt/app/oracle/oradata/PROD/sysaux01.dbf
  3       Full 1410646    19-04-2013 12:35:14 /opt/app/oracle/oradata/PROD/undotbs01.dbf
  4       Full 1410646    19-04-2013 12:35:14 /opt/app/oracle/oradata/PROD/users01.dbf

BS Key  Type LV Size       Device Type Elapsed Time Completion Time    
------- ---- -- ---------- ----------- ------------ -------------------
404     Full    80.00K     DISK        00:00:00     19-04-2013 12:35:59
        BP Key: 407   Status: AVAILABLE  Compressed: NO  Tag: TAG20130419T123513
        Piece Name: /home/oracle/backup/12o7fhdf_1_1
        Keep: BACKUP_LOGS        Until: FOREVER        
  SPFILE Included: Modification time: 19-04-2013 11:18:50
  SPFILE db_unique_name: PROD

BS Key  Size       Device Type Elapsed Time Completion Time    
------- ---------- ----------- ------------ -------------------
426     2.50K      DISK        00:00:00     19-04-2013 12:36:01
        BP Key: 429   Status: AVAILABLE  Compressed: NO  Tag: TAG20130419T123513
        Piece Name: /home/oracle/backup/13o7fhdh_1_1
        Keep: BACKUP_LOGS        Until: FOREVER        

  List of Archived Logs in backup set 426
  Thrd Seq     Low SCN    Low Time            Next SCN   Next Time
  ---- ------- ---------- ------------------- ---------- ---------
  1    94      1410634    19-04-2013 12:35:13 1410670    19-04-2013 12:36:00

BS Key  Type LV Size       Device Type Elapsed Time Completion Time    
------- ---- -- ---------- ----------- ------------ -------------------
427     Full    9.39M      DISK        00:00:02     19-04-2013 12:36:04
        BP Key: 430   Status: AVAILABLE  Compressed: NO  Tag: TAG20130419T123513
        Piece Name: /home/oracle/backup/14o7fhdi_1_1
        Keep: BACKUP_LOGS        Until: FOREVER        
  Control File Included: Ckp SCN: 1410684      Ckp time: 19-04-2013 12:36:02
So I have on file system the archival backups, but I want to copy them into a tape library.
[oracle@vsi08 backup]$ pwd
/home/oracle/backup
[oracle@vsi08 backup]$ ls -lhrt
total 846M
-rw-r----- 1 oracle oinstall 836M Apr 19 12:35 11o7fhc2_1_1
-rw-r----- 1 oracle oinstall  96K Apr 19 12:35 12o7fhdf_1_1
-rw-r----- 1 oracle oinstall 3.0K Apr 19 12:36 13o7fhdh_1_1
-rw-r----- 1 oracle oinstall 9.5M Apr 19 12:36 14o7fhdi_1_1
To complete this task I need to change the backup status to UNAVAILABLE, that is RMAN cannot consider it during a restore and recovery operation: it doesn't exist because I'm going to move it on tape and delete from file system.
RMAN> change backup tag 'TAG20130419T123513' unavailable;

changed backup piece unavailable
backup piece handle=/home/oracle/backup/11o7fhc2_1_1 RECID=31 STAMP=813155714
changed backup piece unavailable
backup piece handle=/home/oracle/backup/12o7fhdf_1_1 RECID=32 STAMP=813155759
changed backup piece unavailable
backup piece handle=/home/oracle/backup/13o7fhdh_1_1 RECID=33 STAMP=813155761
changed backup piece unavailable
backup piece handle=/home/oracle/backup/14o7fhdi_1_1 RECID=34 STAMP=813155763
Changed 4 objects to UNAVAILABLE status
The status is now UNAVAILABLE and I can copy the archival backups on tape.
RMAN> list backup;

List of Backup Sets
===================

BS Key  Type LV Size       Device Type Elapsed Time Completion Time    
------- ---- -- ---------- ----------- ------------ -------------------
403     Full    835.42M    DISK        00:00:44     19-04-2013 12:35:58
        BP Key: 406   Status: UNAVAILABLE  Compressed: NO  Tag: TAG20130419T123513
        Piece Name: /home/oracle/backup/11o7fhc2_1_1
        Keep: BACKUP_LOGS        Until: FOREVER        
  List of Datafiles in backup set 403
  File LV Type Ckp SCN    Ckp Time            Name
  ---- -- ---- ---------- ------------------- ----
  1       Full 1410646    19-04-2013 12:35:14 /opt/app/oracle/oradata/PROD/system01.dbf
  2       Full 1410646    19-04-2013 12:35:14 /opt/app/oracle/oradata/PROD/sysaux01.dbf
  3       Full 1410646    19-04-2013 12:35:14 /opt/app/oracle/oradata/PROD/undotbs01.dbf
  4       Full 1410646    19-04-2013 12:35:14 /opt/app/oracle/oradata/PROD/users01.dbf

BS Key  Type LV Size       Device Type Elapsed Time Completion Time    
------- ---- -- ---------- ----------- ------------ -------------------
404     Full    80.00K     DISK        00:00:00     19-04-2013 12:35:59
        BP Key: 407   Status: UNAVAILABLE  Compressed: NO  Tag: TAG20130419T123513
        Piece Name: /home/oracle/backup/12o7fhdf_1_1
        Keep: BACKUP_LOGS        Until: FOREVER        
  SPFILE Included: Modification time: 19-04-2013 11:18:50
  SPFILE db_unique_name: PROD

BS Key  Size       Device Type Elapsed Time Completion Time    
------- ---------- ----------- ------------ -------------------
426     2.50K      DISK        00:00:00     19-04-2013 12:36:01
        BP Key: 429   Status: UNAVAILABLE  Compressed: NO  Tag: TAG20130419T123513
        Piece Name: /home/oracle/backup/13o7fhdh_1_1
        Keep: BACKUP_LOGS        Until: FOREVER        

  List of Archived Logs in backup set 426
  Thrd Seq     Low SCN    Low Time            Next SCN   Next Time
  ---- ------- ---------- ------------------- ---------- ---------
  1    94      1410634    19-04-2013 12:35:13 1410670    19-04-2013 12:36:00

BS Key  Type LV Size       Device Type Elapsed Time Completion Time    
------- ---- -- ---------- ----------- ------------ -------------------
427     Full    9.39M      DISK        00:00:02     19-04-2013 12:36:04
        BP Key: 430   Status: UNAVAILABLE  Compressed: NO  Tag: TAG20130419T123513
        Piece Name: /home/oracle/backup/14o7fhdi_1_1
        Keep: BACKUP_LOGS        Until: FOREVER        
  Control File Included: Ckp SCN: 1410684      Ckp time: 19-04-2013 12:36:02
Now it could be possible to copy on a tape and remove from file system these archival backups.

That's all.

17 comments:

Anonymous said...

Helpful information. Fortunate me I discovered your website by chance, and
I am stunned why this twist of fate didn't took place in advance! I bookmarked it.

my blog :: joshua pellicer tao of badass ebook

Anonymous said...

I do not even understand how I stopped up right here, but I assumed this put up was great.
I don't know who you might be but definitely you're going to a
well-known blogger should you are not already. Cheers!


Also visit my webpage :: dating guide for women

Anonymous said...

Your means of describing everything in this post is truly pleasant, every
one be capable of without difficulty know it, Thanks a lot.


Here is my weblog - learn to attract women

Anonymous said...

What's up, I read your new stuff regularly. Your humoristic style is witty, keep doing what you're doing!


Here is my site: tao of the badass pdf

Anonymous said...

This is a great tip especially to those fresh to the blogosphere.
Brief but very accurate information… Thanks for
sharing this one. A must read article!

my web blog :: bodipedic 2-inch twin xl-size memory foam mattress topper

Anonymous said...

Ӏ have beеn bгοwsing online more
than 3 hours todaу, yet Ӏ neveг found anу inteгesting aгtiсlе lіke yοurs.
It is pгеtty worth enough fοr mе.

In mу view, if all website owners and bloggers maԁе good cоntеnt as you ԁіd, thе net will be a lot more
usеful thаn eѵer befοre.


Feel freе to visіt my page homeopathy
my site - hcg diet dangers

Anonymous said...

There are not many web sites with information like this man!
Bookmarked!

Also visit my web blog ... Why Am I Not Getting Pregnant

Anonymous said...

With havіn ѕo much сontent anԁ artіcles do you eѵеr run into any pгoblems of plagorism
or cоpyгight νiolation? My site hаs а lot of eхclusіve content
I've either authored myself or outsourced but it seems a lot of it is popping it up all over the internet without my authorization. Do you know any methods to help protect against content from being stolen? I'd genuinely apрreciate it.


Ϻy web blog; do hcg drops work
My web page - hcg tablets

Alphonse said...

This is cool!

oakleyses said...

oakley sunglasses, prada handbags, oakley sunglasses, longchamp handbags, longchamp handbags, louboutin shoes, louis vuitton handbags, coach factory outlet, tiffany and co, coach purses, louis vuitton outlet, polo ralph lauren outlet, air max, prada outlet, longchamp outlet, oakley sunglasses cheap, ray ban sunglasses, louboutin outlet, michael kors outlet, michael kors outlet, tiffany and co, burberry outlet, christian louboutin shoes, coach outlet store online, jordan shoes, polo ralph lauren outlet, louboutin, kate spade handbags, michael kors outlet, coach outlet, air max, gucci outlet, michael kors outlet, ray ban sunglasses, chanel handbags, michael kors outlet, tory burch outlet, nike free, kate spade outlet, louis vuitton outlet, burberry outlet, louis vuitton outlet stores, louis vuitton, nike shoes, michael kors outlet

oakleyses said...

air max, hollister, true religion outlet, nike blazer, louboutin, ray ban sunglasses, polo ralph lauren, michael kors, true religion jeans, sac guess, sac longchamp, hogan outlet, ralph lauren, vans pas cher, sac louis vuitton, air max pas cher, nike free pas cher, nike free, air max, mulberry, nike roshe run, sac burberry, hollister, vanessa bruno, louis vuitton, lululemon, michael kors pas cher, oakley pas cher, air jordan, ray ban pas cher, new balance pas cher, polo lacoste, converse pas cher, north face, sac louis vuitton, michael kors, sac hermes, nike tn, timberland, louis vuitton uk, longchamp, true religion jeans, nike air max, air force, north face

oakleyses said...

abercrombie and fitch, instyler, ghd, bottega veneta, ugg boots, jimmy choo outlet, soccer shoes, ugg pas cher, herve leger, beats by dre, birkin bag, abercrombie and fitch, north face jackets, soccer jerseys, mont blanc, rolex watches, lululemon outlet, celine handbags, nike roshe run, nike trainers, giuseppe zanotti, hollister, wedding dresses, nike huarache, mcm handbags, vans shoes, chi flat iron, babyliss pro, north face outlet, nike roshe, ugg australia, ugg, marc jacobs, barbour, nfl jerseys, p90x, new balance shoes, asics running shoes, ferragamo shoes, mac cosmetics, insanity workout, uggs outlet, reebok outlet, longchamp, valentino shoes

oakleyses said...

converse, air max, gucci, canada goose, juicy couture outlet, canada goose, wedding dresses, moncler, ralph lauren, lancel, montre homme, moncler, louboutin, oakley, karen millen, vans, coach outlet store online, air max, canada goose jackets, ugg, hollister clothing store, louis vuitton, baseball bats, hollister, rolex watches, juicy couture outlet, iphone 6 cases, canada goose uk, canada goose outlet, ugg, moncler, moncler outlet, timberland boots, hollister, supra shoes, moncler, canada goose, converse shoes, toms shoes, moncler, moncler, canada goose, ugg boots, ray ban, parajumpers, canada goose

Mike Smith said...

Tom Ford, the driving force of Gucci's success, is always ahead of the game. With the cat eye trend in full swing, Tom Ford went with the even more dramatic Nico design. The oversize butterfly shape and acetate style with gradient lenses come in 4 warby parker colors, make everyday errands a black tie affair for your eyes. Take a peek into the slideshow for his racy ad campaign photo.For trendy retrophiliacs, The Fendi Spy Sunglasses may be the answer. Sleek metal and tortoise frames resemble old fashioned wood grain and encase colorful gradient lenses, in a modern aviator style design.Get cheap oakley sunglasses a glimpse of supermodel Karlie Kloss wearing Fendi Spy sunglasses featuring an olive green lens, and for more sunglasses in a variety of colors visit Fendi.com
Sunglasses are especially guilty of being overpriced to the point that ski bums, dirt-bag mountain bikers, and weekend warriors can never hope to afford them. But one sunglass oakley glasses frames company at the Outdoor Retailer Summer Market has stepped up to make quality eye wear at prices everyone can afford.Tifosi Optics is a small operation based in Watkinsville, Georgia. Their mission is appropriate for their small-town roots: to make sunglasses for the rest of us. They offer polarized, photochromic interchangeable lenses in various styles http://www.warbyparkersunglasshut.com that would make any Oakley snob link twice before dropping hundreds of dollars. Tifosi's sunglasses are priced from $30 to $90 and all their products come with a lifetime warranty.Some of Tifosi's more compelling features were how easy the lenses were to switch out, and how they are vented with small holes at the oakley replacement lens top. After trying some of the sunglasses on, they were very comfortable due to the adjustable nose and ear pieces that are totally customizable. The frames were also impressive as they are made from a flexible nylon which seemed impossible to break.
Also, if you so choose, the lenses come in a vented option. oakley store This is a must for cyclist. The vents are not just a gimmick; they actually keep the fogging to a minimum. The only time I fog up is when I am climbing at 2 miles an hour and sweating like I was in Swedish sauna.They hug the face quit efficiently and the aforementioned vents oakley online store let air in to keep them clear. And the vents don’t shoot air into your eyes; it’s more of a gentle circulation of freshness. What I really like as a cyclist is that the lenses are tall and, when my head is down, I can still look up through the glasses. Other glasses have oakley sunglasses gotten in the way. I am not a fashion-ista, nor am I sure what that is. Style points, I feel, are completely subjective. So you decide if they look cool or not. All 33 miners in Chile have been rescued following 69 agonizing days of uncertainty regarding their fate; a glorious moment for the oakley outlet workers, friends,

Mike Smith said...

The Nike Flyknit Chukka was born with the Nike Lunar cushioning on bottom. And even though nike free mens it stayed in that mode for quite a while, you had to figure it was only a nike free 3.0 matter of time before the sneakers switched things up in terms of the platform on bottom. Indeed free runs nike 2014 has brought us the Nike Free Flyknit Chukka, an appropriate model considering that the Nike Free nike free run 3 category is celebrating its tenth birthday this year. Gathered here are the latest Nike Free Flyknit Chukka red nike free run womens releases that are around the corner, so check them out below and watch for info on their nike free trainer US arrival right here.
Nike’s rich history of football/soccer footwear hasn’t too often expanded beyond its original nike free 5.0+ purposes to really take hold in the lifestyle realm.

Nike Free said...

The reason I bring up that point is because the public tends to make a judgment when they watch a video involving Police. For example a video taken in San Francisco where an armed man had shot at officers while running away from them but the oakley frogskins only video released was of the suspect lying in a pool of blood after an officer or officers had shot him. The public generally never sees the video that shows the confrontation leading up to the non lethal or lethal action. So, officers are usually judged negatively by the public for their action or oakley lack of action.With officer mounted cameras those judgments can be changed in a moment. For some smaller agencies the camera capturing a video from the officer’s perspective may even save the agency, city or county money in overtime or officer(s) time lost. Once “the client” has knowledge of a video showing the behavior during oakley standard issue an encounter “the client” can then make the “proper” decision.
"Luxottica today is the leading expert around the world in eyewear for women," Olivet said. "So we think their expertise, their direct work with the best designers around the world and their understanding of women's shopping patterns and needs can only help us get fake oakleys better at what we've started to do.Oakley Inc. said it would merge its half a dozen sunglasses outlets into a single brand as it focuses on optics.Most of the Foothill Ranch company's 110 mall-based stores would be consolidated under its Sunglass Icon brand.The change affects Oakley's Occhiali da Sole, Optica Sporting Eyes, Sunglass Club sunglass hut and Sunglass Designs chains.
Also included is a scratch-resistant outer coating and an inner coating that reduces fog.Oakley’s Prizm lens technology delivers increased contrast, reduced eye fatigue and faster target acquisition in a shooting environment. Prizm lenses use formulated dye-compounding technology to enhance specific wavelengths of color while muting others so that targets oakley prescription glasses are accentuated and background visuals like trees and dirt are subdued, in both bright and low-light conditions.In order for the SI Ballistic Det Cord to meet the MIL PRF 32432 ballistic impact requirement, the glasses must be hit with a 0.15 caliber, 5.85 grain, T37 shaped projectile at approximately 640-660 feet per second, without oakley store the glasses’ lenses fracturing or any dislodged components. This test mimics the hazardous conditions the user might experience if exposed to ballistic fragmentation.
Brillhart said he’s able to offer such deals for a few reasons. First, the company spends little money on marketing as the majority of his sales are due to word-of-mouth oakleys from satisfied customers. Second, there is no overhead. There are two men behind the glasses: the manufacturer, who makes the frames in Osaka, Japan, and Brillhart, the creator. "The Long Beach Fire Department was a great market," said Brillhart. "The problem is, they don't order more because the glasses never break! "Most often when oakley military we go to war/battle or as a SWAT Officer to the JOB,

roba. gad3 said...

http://caravank.com/factory/
http://caravank.com/for-sale-in-riyadh/
http://caravank.com/withdrawn-for-sale/
http://caravank.com/withdrawal-rooms/
http://caravank.com/us-for-sale/
http://caravank.com/car-caravan/
http://caravank.com/for-sale-in-jeddah/