Tuesday, February 18, 2014

How to solve the error 'ERROR: NMO Not Setuid-root (Unix-only)' on Oracle Enterprise Manager Cloud Control while contacting the EM Agent

I've just finished to setup at home an Oracle Real Application Clusters virtual environment formed by two nodes.

I would like to monitor and eventually execute some commands from another machine where I've installed the latest (at the moment of this post of course) available Enterprise Manager Cloud Console 12c that is Oracle Enterprise Manager Cloud Control 12c Release 3 Plug-in Update 1 (12.1.0.3) which contains the original 12c Release 3 (12.1.0.3) software binaries along with a few new plug-ins and updated plug-in versions.

I would like to compare the easy way to manage a Real Application Clusters from a GUI solution against the classic command line interface (CLI).

After I successfully deployed the agents from my Enterprise Manager Clod Control machine to the two target RAC nodes whenever I tried to perform some operations they resulted in the following error:
ERROR: NMO not setuid-root (Unix-only)

The following steps helped me to solve the problem.

On my nodes (demorac01 and demorac02) I created a local directory where EMCC deployed the agent. Let's have a look at the content of that directory in the second node:
[oracle@demorac02 agent12cr3]$ pwd
/u01/app/oracle/agent12cr3
[oracle@demorac02 agent12cr3]$ ll
total 28
-rw-rw-r--  1 oracle oinstall  178 Jun 22  2013 agentimage.properties
drwxr-xr-x  6 oracle oinstall 4096 Feb 15 10:36 agent_inst
drwxr-xr-x  3 oracle oinstall 4096 Feb 14 10:34 core
drwxr-xr-x 13 oracle oinstall 4096 Feb 14 23:04 plugins
-rwxr-xr-x  1 oracle oinstall  449 Feb 14 10:32 plugins.txt
-rw-r--r--  1 oracle oinstall  599 Feb 14 10:38 plugins.txt.status
drwxr-xr-x  5 oracle oinstall 4096 Feb 14 10:36 sbin
Inside the sbin directory there are other files that according to the Oracle documentations should be owned by root. During my local deploy they were set up to be owned by oracle user instead.
[oracle@demorac02 agent12cr3]$ cd sbin
[oracle@demorac02 sbin]$ ll
total 1372
drwxr-xr-x  3 oracle oinstall   4096 Feb 14 10:36 cfgtoollogs
drwxr-xr-x  3 oracle oinstall   4096 Jun 22  2013 install
drwxrwx--- 11 oracle oinstall   4096 Jun 22  2013 inventory
-rwxr-xr-x  1 oracle oinstall  22754 Jun 22  2013 nmb.0
-rwxr-xr-x  1 oracle oinstall  43494 Jun 22  2013 nmgsshe
-rwxr-xr-x  1 oracle oinstall  43494 Jun 22  2013 nmgsshe.0
-rwxr-xr-x  1 oracle oinstall  57835 Jun 22  2013 nmhs.0
-rwxr-xr-x  1 oracle oinstall  40007 Jun 22  2013 nmo.0
-rwxr-xr-x  1 oracle oinstall 585242 Jun 22  2013 nmosudo
-rwxr-xr-x  1 oracle oinstall 585242 Jun 22  2013 nmosudo.0
-rw-r-----  1 oracle oinstall     56 Jan 26 16:41 oraInst.loc
Before proceding with the solution adopted just note that the agent was running and collecting all the available info from the RAC node as you can see from the command below:
[oracle@demorac02 sbin]$ cd /u01/app/oracle/agent12cr3/core/12.1.0.3.0/bin
[oracle@demorac02 bin]$ ./emctl status agent
Oracle Enterprise Manager Cloud Control 12c Release 3  
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version     : 12.1.0.3.0
OMS Version       : 12.1.0.3.0
Protocol Version  : 12.1.0.1.0
Agent Home        : /u01/app/oracle/agent12cr3/agent_inst
Agent Binaries    : /u01/app/oracle/agent12cr3/core/12.1.0.3.0
Agent Process ID  : 21139
Parent Process ID : 21096
Agent URL         : https://192.168.56.102:3872/emd/main/
Repository URL    : https://emcc.example.com:4900/empbs/upload
Started at        : 2014-02-14 10:39:20
Started by user   : oracle
Last Reload       : (none)
Last successful upload                       : 2014-02-15 10:30:47
Last attempted upload                        : 2014-02-15 10:30:47
Total Megabytes of XML files uploaded so far : 1.3
Number of XML files pending upload           : 0
Size of XML files pending upload(MB)         : 0
Available disk space on upload filesystem    : 29.49%
Collection Status                            : Collections enabled
Heartbeat Status                             : Ok
Last attempted heartbeat to OMS              : 2014-02-15 10:35:48
Last successful heartbeat to OMS             : 2014-02-15 10:35:48
Next scheduled heartbeat to OMS              : 2014-02-15 10:36:48

---------------------------------------------------------------
Agent is Running and Ready
To correct the permissions on the desired files you have to run again the file root.sh located under the software location of the Enterprise Manager Cloud Control.
[oracle@demorac02 sbin]$ su - 
Password: 
[root@demorac02 ~]# cd /u01/app/oracle/agent12cr3/core/12.1.0.3.0/
[root@demorac02 12.1.0.3.0]# ll
total 180
drwxr-xr-x  4 oracle oinstall 4096 Jun 22  2013 archives
drwxr-xr-x  2 oracle oinstall 4096 Feb 14 10:36 bin
drwxr-xr-x  6 oracle oinstall 4096 Feb 14 10:39 cfgtoollogs
drwxr-xr-x  6 oracle oinstall 4096 Jun 22  2013 common
drwxr-xr-x  3 oracle oinstall 4096 Jun 22  2013 diagnostics
-rw-r-----  1 oracle oinstall  430 Apr  7  2008 dummy.ic.linux.txt
-rw-r-----  1 oracle oinstall  412 Apr  7  2008 dummy.ic.txt
drwxrwxr-x  3 oracle oinstall 4096 Jun 22  2013 EMStage
drwxr-xr-x  6 oracle oinstall 4096 Feb 14 10:36 install
drwxr-xr-x  2 oracle oinstall 4096 Jun 22  2013 instantclient
drwxrwx--- 14 oracle oinstall 4096 Jun 22  2013 inventory
drwxr-xr-x  3 oracle oinstall 4096 Jun 22  2013 jdbc
drwxr-xr-x  6 oracle oinstall 4096 Feb 14 10:34 jdk
drwxr-xr-x  2 oracle oinstall 4096 Feb 14 10:34 jlib
drwxr-xr-x  2 oracle oinstall 4096 Jun 22  2013 js
drwxr-xr-x  2 oracle oinstall 4096 Jun 22  2013 jythonLib
drwxr-xr-x  5 oracle oinstall 4096 Jun 22  2013 ldap
drwxr-xr-x  4 oracle oinstall 4096 Jun 22  2013 lib
drwxr-xr-x  3 oracle oinstall 4096 Jun 22  2013 lib32
drwxr-xr-x  4 oracle oinstall 4096 Jun 22  2013 modules
drwxr-xr-x  3 oracle oinstall 4096 Jun 22  2013 network
drwxr-xr-x  3 oracle oinstall 4096 Jun 22  2013 ocm
drwxr-xr-x 10 oracle oinstall 4096 Feb 14 10:35 OPatch
drwxr-xr-x  7 oracle oinstall 4096 Jun 22  2013 opmn
-rw-r-----  1 oracle oinstall   56 Jan 26 16:41 oraInst.loc
drwxr-xr-x  7 oracle oinstall 4096 Feb 14 10:35 oui
drwxr-xr-x  5 oracle oinstall 4096 Feb 14 10:35 perl
drwxr-xr-x  2 oracle oinstall 4096 Jun 22  2013 prereqs
drwxr-xr-x 10 oracle oinstall 4096 Jun 22  2013 rda
drwxr-xr-x  3 oracle oinstall 4096 Jun 22  2013 relnotes
-rwxr-xr-x  1 oracle oinstall 1991 Feb 14 10:35 replacebins.sh
-rwxr-xr-x  1 oracle oinstall 1986 Jun 22  2013 replacebins.sh.ouibak
-rwxr-xr-x  1 oracle oinstall 1968 Jun 20  2012 replacebins.sh.template
-rw-r--r--  1 oracle oinstall 1968 Jun 20  2012 replacebins.sh.template.ouibak
-rwxr-xr-x  1 oracle oinstall 4079 Feb 14 10:35 root.sh
-rwxr-xr-x  1 oracle oinstall 3933 Jun 22  2013 root.sh.old
-rwxr-x---  1 oracle oinstall   10 Feb 14 10:34 root.sh.old.1
-rwxr-x---  1 oracle oinstall   10 Feb 14 10:35 root.sh.ouibak
-rwxr-xr-x  1 oracle oinstall 3874 Feb 14 10:35 root.sh.template
-rw-r--r--  1 oracle oinstall 3869 Jun 22  2013 root.sh.template.ouibak
drwxr-xr-x  5 oracle oinstall 4096 Jun 22  2013 stage
drwxr-xr-x  6 oracle oinstall 4096 Jun 22  2013 sysman
drwxr-xr-x  3 oracle oinstall 4096 Jun 22  2013 ucp
drwxr-xr-x  5 oracle oinstall 4096 Jun 22  2013 xdk
drwxr-xr-x  2 oracle oinstall 4096 Jun 22  2013 xsds
That file needs to be run as root user too.
[root@demorac02 12.1.0.3.0]# ./root.sh
Finished product-specific root actions.
/etc exist

Creating /etc/oragchomelist file...
Finished product-specific root actions.
Now if you go again under the previous sbin directory you can see that the owner is changed:
[root@demorac02 12.1.0.3.0]# cd /u01/app/oracle/agent12cr3/sbin/
[root@demorac02 sbin]# ll
total 1496
drwxr-xr-x  3 oracle oinstall   4096 Feb 14 10:36 cfgtoollogs
drwxr-xr-x  3 oracle oinstall   4096 Jun 22  2013 install
drwxrwx--- 11 oracle oinstall   4096 Jun 22  2013 inventory
-rws--x---  1 root   oinstall  22754 Feb 15 10:48 nmb
-rwxr-xr-x  1 oracle oinstall  22754 Jun 22  2013 nmb.0
-rwxr-xr-x  1 root   oinstall  43494 Feb 15 10:48 nmgsshe
-rwxr-xr-x  1 oracle oinstall  43494 Jun 22  2013 nmgsshe.0
-rws--x---  1 root   oinstall  57835 Feb 15 10:48 nmhs
-rwxr-xr-x  1 oracle oinstall  57835 Jun 22  2013 nmhs.0
-rws--x---  1 root   oinstall  40007 Feb 15 10:48 nmo
-rwxr-xr-x  1 oracle oinstall  40007 Jun 22  2013 nmo.0
-rwxr-xr-x  1 root   oinstall 585242 Feb 15 10:48 nmosudo
-rwxr-xr-x  1 oracle oinstall 585242 Jun 22  2013 nmosudo.0
-rw-r-----  1 oracle oinstall     56 Jan 26 16:41 oraInst.loc
It's time to bounce the agent so the new settings could be used by the Enterprise Manager Clod Control during its operations. You must stop and start the agent as oracle user.
[oracle@demorac02 bin]$ id
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),492(vboxsf),54322(dba)
[oracle@demorac02 bin]$ pwd
/u01/app/oracle/agent12cr3/core/12.1.0.3.0/bin
[oracle@demorac02 bin]$ ./emctl stop agent
Oracle Enterprise Manager Cloud Control 12c Release 3  
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
Stopping agent ..... stopped.
[oracle@demorac02 bin]$ ./emctl start agent
Oracle Enterprise Manager Cloud Control 12c Release 3  
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
Starting agent ............................................................................... started.
[oracle@demorac02 bin]$ ./emctl status agent
Oracle Enterprise Manager Cloud Control 12c Release 3  
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version     : 12.1.0.3.0
OMS Version       : 12.1.0.3.0
Protocol Version  : 12.1.0.1.0
Agent Home        : /u01/app/oracle/agent12cr3/agent_inst
Agent Binaries    : /u01/app/oracle/agent12cr3/core/12.1.0.3.0
Agent Process ID  : 11011
Parent Process ID : 10961
Agent URL         : https://192.168.56.102:3872/emd/main/
Repository URL    : https://emcc.example.com:4900/empbs/upload
Started at        : 2014-02-15 10:51:36
Started by user   : oracle
Last Reload       : (none)
Last successful upload                       : 2014-02-15 10:53:23
Last attempted upload                        : 2014-02-15 10:53:23
Total Megabytes of XML files uploaded so far : 0.01
Number of XML files pending upload           : 0
Size of XML files pending upload(MB)         : 0
Available disk space on upload filesystem    : 29.48%
Collection Status                            : Collections enabled
Heartbeat Status                             : Ok
Last attempted heartbeat to OMS              : 2014-02-15 10:55:37
Last successful heartbeat to OMS             : 2014-02-15 10:55:37
Next scheduled heartbeat to OMS              : 2014-02-15 10:56:38

---------------------------------------------------------------
Agent is Running and Ready
I have a RAC with two nodes so the same steps need to be executed on the other node, demorac01 in my case. Here there are the wrong settings before running the root.sh script.
[oracle@demorac01 sbin]$ pwd
/u01/app/oracle/agent12cr3/sbin
[oracle@demorac01 sbin]$ ll
total 1372
drwxr-xr-x  3 oracle oinstall   4096 Feb 14 10:21 cfgtoollogs
drwxr-xr-x  3 oracle oinstall   4096 Jun 22  2013 install
drwxrwx--- 11 oracle oinstall   4096 Jun 22  2013 inventory
-rwxr-xr-x  1 oracle oinstall  22754 Jun 22  2013 nmb.0
-rwxr-xr-x  1 oracle oinstall  43494 Jun 22  2013 nmgsshe
-rwxr-xr-x  1 oracle oinstall  43494 Jun 22  2013 nmgsshe.0
-rwxr-xr-x  1 oracle oinstall  57835 Jun 22  2013 nmhs.0
-rwxr-xr-x  1 oracle oinstall  40007 Jun 22  2013 nmo.0
-rwxr-xr-x  1 oracle oinstall 585242 Jun 22  2013 nmosudo
-rwxr-xr-x  1 oracle oinstall 585242 Jun 22  2013 nmosudo.0
-rw-r-----  1 oracle oinstall     56 Jan 26 16:41 oraInst.loc
Run the root.sh script on demorac01 as root user.
[oracle@demorac01 sbin]$ su -
Password: 
[root@demorac01 ~]# cd /u01/app/oracle/agent12cr3/core/12.1.0.3.0/
[root@demorac01 12.1.0.3.0]# ./root.sh
Finished product-specific root actions.
/etc exist

Creating /etc/oragchomelist file...
Finished product-specific root actions.
Let's check the new settings.
[root@demorac01 12.1.0.3.0]# cd ../../sbin/
[root@demorac01 sbin]# ll
total 1496
drwxr-xr-x  3 oracle oinstall   4096 Feb 14 10:21 cfgtoollogs
drwxr-xr-x  3 oracle oinstall   4096 Jun 22  2013 install
drwxrwx--- 11 oracle oinstall   4096 Jun 22  2013 inventory
-rws--x---  1 root   oinstall  22754 Feb 15 10:54 nmb
-rwxr-xr-x  1 oracle oinstall  22754 Jun 22  2013 nmb.0
-rwxr-xr-x  1 root   oinstall  43494 Feb 15 10:54 nmgsshe
-rwxr-xr-x  1 oracle oinstall  43494 Jun 22  2013 nmgsshe.0
-rws--x---  1 root   oinstall  57835 Feb 15 10:54 nmhs
-rwxr-xr-x  1 oracle oinstall  57835 Jun 22  2013 nmhs.0
-rws--x---  1 root   oinstall  40007 Feb 15 10:54 nmo
-rwxr-xr-x  1 oracle oinstall  40007 Jun 22  2013 nmo.0
-rwxr-xr-x  1 root   oinstall 585242 Feb 15 10:54 nmosudo
-rwxr-xr-x  1 oracle oinstall 585242 Jun 22  2013 nmosudo.0
-rw-r-----  1 oracle oinstall     56 Jan 26 16:41 oraInst.loc
Bounce the agent on demorac01: stop and start it as oracle user.
[root@demorac01 sbin]# logout
[oracle@demorac01 sbin]$ cd ../core/12.1.0.3.0/bin/
[oracle@demorac01 bin]$ ./emctl stop agent
Oracle Enterprise Manager Cloud Control 12c Release 3  
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
Stopping agent ..... stopped.
[oracle@demorac01 bin]$ ./emctl start agent
Oracle Enterprise Manager Cloud Control 12c Release 3  
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
Starting agent .......................................................... started.
Let's check the agent status.
[oracle@demorac01 bin]$ ./emctl status agent
Oracle Enterprise Manager Cloud Control 12c Release 3  
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version     : 12.1.0.3.0
OMS Version       : 12.1.0.3.0
Protocol Version  : 12.1.0.1.0
Agent Home        : /u01/app/oracle/agent12cr3/agent_inst
Agent Binaries    : /u01/app/oracle/agent12cr3/core/12.1.0.3.0
Agent Process ID  : 13377
Parent Process ID : 13332
Agent URL         : https://192.168.56.101:3872/emd/main/
Repository URL    : https://emcc.example.com:4900/empbs/upload
Started at        : 2014-02-15 10:56:16
Started by user   : oracle
Last Reload       : (none)
Last successful upload                       : 2014-02-15 10:57:01
Last attempted upload                        : 2014-02-15 10:57:01
Total Megabytes of XML files uploaded so far : 0
Number of XML files pending upload           : 0
Size of XML files pending upload(MB)         : 0
Available disk space on upload filesystem    : 21.39%
Collection Status                            : Collections enabled
Heartbeat Status                             : Ok
Last attempted heartbeat to OMS              : 2014-02-15 10:57:00
Last successful heartbeat to OMS             : 2014-02-15 10:57:00
Next scheduled heartbeat to OMS              : 2014-02-15 10:58:01

---------------------------------------------------------------
Agent is Running and Ready
So both agents are now running again on the RAC nodes. Let's verify that I solved my issue directly from the Enterprise Manager Cloud Control: the error "NMO not setuid-root (Unix-only)" was thrown the first time when I tried to start up a pluggable database.
 

Let's do the same step again. Here is the current situation: a CDB in open mode and the pluggable database PDB001 not open.
 

Click on the target name link named "DEMO.demorac_PDB001". It will open the following screenshot and information: "Database is not open. Regions that fetch real-time data from target database will not be displayed"

So I want to open that PDB. Point your mouse over "Oracle Database" menu, then "Control" and finally "Open/Close Pluggable Database".

Give the credentials to access the remote Oracle database.

It seems to work because the Enterprise Manager Cloud Console is currently showing a new page.
 

Because I want to open the pluggable database PDB001 on both the available instances I select both instances using the CTRL button, and then clicked on "Open" from the Action menu.
 

A new browser page will show the status of the operation in progress.

When the operation terminates successfully the "Open/Close Pluggable Database" page will show that the status of the pluggable database PDB001 is now OPEN.
 

Clicking on the target link of the pluggable database named "DEMO.demorac_PDB001" will show new activities are performed and so collected from the agent.
 

That's all.

11 comments:

Anonymous said...

Thanx a lot pal
tis is really helpful... :)

Lucien Jacquet said...

standalone database (11.2.0.3)
Cloud Control 12c (12.0.2.0)
weekly backup job managed by EMCC
job failed with same error

I followed your instructions as far as executing root.sh (I don't have a RAC environment), verified that the file ownership had changed, and re-ran the failed backup job. Success!

Thanks very much for the detailed, helpful resolution.

oakleyses said...

oakley sunglasses, prada handbags, oakley sunglasses, longchamp handbags, longchamp handbags, louboutin shoes, louis vuitton handbags, coach factory outlet, tiffany and co, coach purses, louis vuitton outlet, polo ralph lauren outlet, air max, prada outlet, longchamp outlet, oakley sunglasses cheap, ray ban sunglasses, louboutin outlet, michael kors outlet, michael kors outlet, tiffany and co, burberry outlet, christian louboutin shoes, coach outlet store online, jordan shoes, polo ralph lauren outlet, louboutin, kate spade handbags, michael kors outlet, coach outlet, air max, gucci outlet, michael kors outlet, ray ban sunglasses, chanel handbags, michael kors outlet, tory burch outlet, nike free, kate spade outlet, louis vuitton outlet, burberry outlet, louis vuitton outlet stores, louis vuitton, nike shoes, michael kors outlet

oakleyses said...

air max, hollister, true religion outlet, nike blazer, louboutin, ray ban sunglasses, polo ralph lauren, michael kors, true religion jeans, sac guess, sac longchamp, hogan outlet, ralph lauren, vans pas cher, sac louis vuitton, air max pas cher, nike free pas cher, nike free, air max, mulberry, nike roshe run, sac burberry, hollister, vanessa bruno, louis vuitton, lululemon, michael kors pas cher, oakley pas cher, air jordan, ray ban pas cher, new balance pas cher, polo lacoste, converse pas cher, north face, sac louis vuitton, michael kors, sac hermes, nike tn, timberland, louis vuitton uk, longchamp, true religion jeans, nike air max, air force, north face

oakleyses said...

abercrombie and fitch, instyler, ghd, bottega veneta, ugg boots, jimmy choo outlet, soccer shoes, ugg pas cher, herve leger, beats by dre, birkin bag, abercrombie and fitch, north face jackets, soccer jerseys, mont blanc, rolex watches, lululemon outlet, celine handbags, nike roshe run, nike trainers, giuseppe zanotti, hollister, wedding dresses, nike huarache, mcm handbags, vans shoes, chi flat iron, babyliss pro, north face outlet, nike roshe, ugg australia, ugg, marc jacobs, barbour, nfl jerseys, p90x, new balance shoes, asics running shoes, ferragamo shoes, mac cosmetics, insanity workout, uggs outlet, reebok outlet, longchamp, valentino shoes

oakleyses said...

converse, air max, gucci, canada goose, juicy couture outlet, canada goose, wedding dresses, moncler, ralph lauren, lancel, montre homme, moncler, louboutin, oakley, karen millen, vans, coach outlet store online, air max, canada goose jackets, ugg, hollister clothing store, louis vuitton, baseball bats, hollister, rolex watches, juicy couture outlet, iphone 6 cases, canada goose uk, canada goose outlet, ugg, moncler, moncler outlet, timberland boots, hollister, supra shoes, moncler, canada goose, converse shoes, toms shoes, moncler, moncler, canada goose, ugg boots, ray ban, parajumpers, canada goose

fouedgray said...

Thanks for this post.

oakleyses said...

jordan shoes, christian louboutin, uggs outlet, michael kors outlet online, uggs on sale, louis vuitton outlet, louis vuitton outlet, louis vuitton, ray ban sunglasses, replica watches, christian louboutin uk, chanel handbags, michael kors outlet online, uggs outlet, longchamp outlet, nike air max, michael kors outlet, burberry handbags, tiffany and co, polo outlet, nike free, nike air max, ugg boots, oakley sunglasses, ray ban sunglasses, michael kors outlet online, oakley sunglasses, christian louboutin outlet, longchamp outlet, prada handbags, gucci handbags, prada outlet, oakley sunglasses wholesale, michael kors outlet, oakley sunglasses, kate spade outlet, christian louboutin shoes, louis vuitton outlet, tory burch outlet, ugg boots, michael kors outlet online, burberry outlet, cheap oakley sunglasses, louis vuitton, ray ban sunglasses, nike outlet, longchamp outlet

oakleyses said...

sac vanessa bruno, new balance, vans pas cher, ray ban uk, nike blazer pas cher, true religion outlet, michael kors outlet, true religion outlet, replica handbags, polo lacoste, oakley pas cher, coach purses, hollister uk, abercrombie and fitch uk, nike free uk, north face uk, louboutin pas cher, polo ralph lauren, hollister pas cher, nike air max uk, michael kors pas cher, nike air max, true religion jeans, timberland pas cher, nike air max uk, coach outlet, air max, michael kors, jordan pas cher, sac hermes, north face, lululemon canada, coach outlet store online, nike roshe, sac longchamp pas cher, nike air force, mulberry uk, hogan outlet, ralph lauren uk, longchamp pas cher, michael kors, converse pas cher, burberry pas cher, nike roshe run uk, true religion outlet, kate spade, nike free run, nike tn, ray ban pas cher, guess pas cher

oakleyses said...

asics running shoes, babyliss, soccer jerseys, hermes belt, reebok outlet, ipad cases, oakley, iphone cases, soccer shoes, iphone 5s cases, nfl jerseys, north face outlet, abercrombie and fitch, ghd hair, vans outlet, iphone 6 cases, hollister, nike roshe run, wedding dresses, mac cosmetics, lululemon, new balance shoes, jimmy choo outlet, instyler, giuseppe zanotti outlet, p90x workout, s6 case, chi flat iron, iphone 6s cases, longchamp uk, baseball bats, mcm handbags, iphone 6 plus cases, bottega veneta, ferragamo shoes, timberland boots, mont blanc pens, insanity workout, nike air max, nike trainers uk, herve leger, nike huaraches, celine handbags, north face outlet, beats by dre, iphone 6s plus cases, valentino shoes, ralph lauren, hollister clothing, louboutin

oakleyses said...

moncler uk, louis vuitton, thomas sabo, wedding dresses, barbour, moncler, gucci, montre pas cher, supra shoes, hollister, barbour uk, nike air max, karen millen uk, pandora uk, moncler, canada goose uk, coach outlet, ugg, juicy couture outlet, swarovski, canada goose, louis vuitton, moncler outlet, louis vuitton, hollister, ugg,ugg australia,ugg italia, canada goose outlet, replica watches, pandora jewelry, ugg,uggs,uggs canada, moncler, canada goose outlet, ugg pas cher, louis vuitton, juicy couture outlet, swarovski crystal, louis vuitton, canada goose, pandora charms, canada goose outlet, links of london, marc jacobs, lancel, converse, converse outlet, toms shoes, doudoune moncler, moncler, pandora jewelry, canada goose jackets, vans, canada goose