Monday, January 27, 2020

How to create signed url on google cloud storage?

A signed URL is a URL that provides limited permission and time to make a request. It's good to be used by someone who does not have a Google Account. I caught up reading on Google Cloud documents and finding how to do it. Assume I would like to share file on google cloud storage to my friend who does have a Google Account. Example: gs://mysurachartbucket/test.txt
[student@centos~]$ gsutil mb gs://mysurachartbucket
Creating gs://mysurachartbucket/...
[student@centos~]$ cat test.txt
TEST
[student@centos~]$  gsutil cp test.txt gs://mysurachartbucket/
Copying file://test.txt [Content-Type=text/plain]...
- [1 files][    5.0 B/    5.0 B]
Operation completed over 1 objects/5.0 B.
[student@centos~]$ gsutil ls gs://mysurachartbucket/test.txt
gs://mysurachartbucket/test.txt
First of all, I need keystore-file from service account. So, To create service account and key file.

[student@centos~]$ gcloud iam service-accounts list
NAME                                    EMAIL                                                DISABLED
Compute Engine default service account  ********-compute@developer.gserviceaccount.com  False
[student@centos~]$ gcloud iam service-accounts create surachart
Created service account [surachart].
[student@centos~]$ gcloud iam service-accounts list
NAME                                    EMAIL                                                DISABLED
Compute Engine default service account  ********-compute@developer.gserviceaccount.com  False
                                        surachart@myproject.iam.gserviceaccount.com         False
[student@centos~]$ gcloud iam service-accounts keys create ~/surachart.json   --iam-account surachart@myproject.iam.gserviceaccount.com
created key [4d6b1bd*********08f966dd31] of type [json] as [/home/student/surachart.json] for [surachart@myproject.iam.gserviceaccount.com]
Then, service account should be able to read file in bucket.
[student@centos~]$ gsutil acl ch  -u surachart@myproject.iam.gserviceaccount.com:R gs://mysurachartbucket/test.txt
Updated ACL on gs://mysurachartbucket/test.txt
Finally, create signed url by using gsutil command.
[student@centos~]$ gsutil signurl -d 20m surachart.json gs://mysurachartbucket/test.txt
CommandException: The signurl command requires the pyopenssl library (try pip install pyopenssl or easy_install pyopenssl)

####As error that need to install pyopenssl.
[student@centos~]$ sudo pip install pyopenssl
[student@centos~]$ gsutil signurl -d 20m surachart.json gs://mysurachartbucket/test.txt
URL     HTTP Method Expiration Signed URL
gs://mysurachartbucket/test.txt GET 2020-01-27 21:34:08 https://storage.googleapis.com/mysurachartbucket/test.txt?x-goog-signature=99dbc749d2891eb1d9d22a5ccd03a81d4f0366380ff3bb0c34faf246d20677290778c6033a81fce43363709b244a882308b1c8590eaed409e1c8a0d4aca76cfec8537b1231e6b1f57************c6abaaacd128ac85f798edfb41bfa48d688897882be28cd1838520144ff197a5e84f499da914c2f8b309c32343011974a8f888163cba2a33c491fd858906bce2ad3cb5c5249c1e79127d200dccea553deafe7e1eb43a8b1527cb20e935c66129b0cad1683f01b6474a4c2940b92dd6daaa65da48fba7cbe94ed5881d46f268908735b2ad12ef2b1f7b0e79a2dd4a527cc611ea35718db96db&x-goog-algorithm=GOOG4-RSA-SHA256&x-goog-credential=surachart%40myproject.iam.gserviceaccount.com%2F20200127%2Fus%2Fstorage%2Fgoog4_request&x-goog-date=20200127T140408Z&x-goog-expires=1800&x-goog-signedheaders=host
This signed url will expire in 20 minutes. Then send it to my friend.

Reference: 
https://cloud.google.com/storage/docs/access-control/signed-urls
https://cloud.google.com/storage/docs/gsutil/commands/signurl

Monday, October 22, 2018

Oracle Database EM 18 XE Available to Remote Clients

I found lot of posts about Oracle Database 18 XE. It's very interesting for me.  I didn't blog about how to install, because it's very easy for using rpm package and document very helpful.
I was interested in Enterprise Manager Database Express 18.4.0.0.0. How it looks like?
- Installing. I used CentOS7.
[student@centos-learning ~]$ sudo yum -y localinstall oracle-database*18c*
[student@centos-learning ~]$ sudo rpm -qa |grep oracle
oracle-database-preinstall-18c-1.0-1.el7.x86_64
oracle-database-xe-18c-1.0-1.x86_64

[student@centos-learning ~]$ sudo /etc/init.d/oracle-xe-18c configure
Specify a password to be used for database accounts. Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9]. Note that the same password will be used for SYS, SYSTEM and PDBADMIN accounts:
The password you entered contains invalid characters. Enter password:
Confirm the password:
Configuring Oracle Listener.
Listener configuration succeeded.
Configuring Oracle Database XE.
Enter SYS user password:
*********
Enter SYSTEM user password:
********
Enter PDBADMIN User Password:
*********
Prepare for db operation
7% complete
Copying database files
29% complete
Creating and starting Oracle instance
30% complete
31% complete
34% complete
38% complete
41% complete
43% complete
Completing Database Creation
47% complete
50% complete
Creating Pluggable Databases
54% complete
71% complete
Executing Post Configuration Actions
93% complete
Running Custom Scripts
100% complete
Database creation complete. For details check the logfiles at:
 /opt/oracle/cfgtoollogs/dbca/XE.
Database Information:
Global Database Name:XE
System Identifier(SID):XE
Look at the log file "/opt/oracle/cfgtoollogs/dbca/XE/XE.log" for further details.
Connect to Oracle Database using one of the connect strings:
     Pluggable database: centos-learning.surachartopun.com/XEPDB1
     Multitenant container database: centos-learning.surachartopun.com
Use https://localhost:5500/em to access Oracle Enterprise Manager for Oracle Database XE
[student@centos-learning ~]$ netstat -ltn |grep 5500
tcp        0      0 127.0.0.1:5500          0.0.0.0:*               LISTEN
- As I didn't want to connect 127.0.0.1, I changed binding - "Making Oracle Database EM Express Available to Remote Clients"
SQL> !netstat -ltn |grep 5500
tcp        0      0 127.0.0.1:5500          0.0.0.0:*               LISTEN
SQL> !lsnrctl status | grep HTTP
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=127.0.0.1)(PORT=5500))(Security=(my_wallet_directory=/opt/oracle/product/18c/dbhomeXE/admin/XE/xdb_wallet))(Presentation=HTTP)(Session=RAW))
SQL>
SQL>
SQL> EXEC DBMS_XDB.SETLISTENERLOCALACCESS(FALSE);
PL/SQL procedure successfully completed.
SQL> !netstat -ltn |grep 5500
tcp        0      0 0.0.0.0:5500            0.0.0.0:*               LISTEN
SQL> !lsnrctl status | grep HTTP
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=centos-learning.surachartopun.com)(PORT=5500))(Security=(my_wallet_directory=/opt/oracle/admin/XE/xdb_wallet))(Presentation=HTTP)(Session=RAW))
- Browsed it - https://IP:5500/em

However, I got some error like "Connection with database failed. Database instance might be down."

- Checked alert log file and fixed.
Error: Global ports off in Root, do SetGlobalPortEnabled(TRUE) to enable.
bash-4.2$ tail -f alert_XE.log
2018-10-22T22:06:32.890217+07:00
Global ports off in Root, do SetGlobalPortEnabled(TRUE) to enable.
2018-10-22T22:06:38.489011+07:00
Global ports off in Root, do SetGlobalPortEnabled(TRUE) to enable.
2018-10-22T22:10:32.402822+07:00
Resize operation completed for file# 3, old size 501760K, new size 512000K
2018-10-22T22:15:55.791490+07:00
Global ports off in Root, do SetGlobalPortEnabled(TRUE) to enable.
2018-10-22T22:18:02.248906+07:00
Global ports off in Root, do SetGlobalPortEnabled(TRUE) to enable.
Setting the Global Port for EM Express to Manage a CDB and the PDBs. (It might not be the right solution, but I just wanted to see EM).
SQL> select dbms_xdb_config.getHttpsPort() from dual;
DBMS_XDB_CONFIG.GETHTTPSPORT()
------------------------------
                          5500
SQL> exec dbms_xdb_config.SetGlobalPortEnabled(TRUE)
PL/SQL procedure successfully completed.
- Login again.

It worked fine for now.

Reference: Installation Guide for Linux x86-64

Wednesday, September 19, 2018

How to allow private connectivity across organizations(GCP)?

It's interesting, when you would like to allow private connectivity across two VPC networks that they belong to the different project/organization on Google Cloud Platform (GCP).

As google document that we can use VPC Network Peering? VPC Network Peering is a decentralized or distributed approach to multi-project networking. Additional, it works with Compute Engine, Kubernetes Engine,and App Engine flexible environments.

I did a lab about Virtual Private Cloud (VPC) Network Peering. There shows to do VPC Network Peering between VPC networks in the same project. So, I would like to see how it works on across organizations.

Example: On My Organization (my project), I would like to connect server (Private IP Address) on another Organization (another project).

As a subnet CIDR prefix in one peered VPC network cannot overlap with a subnet CIDR prefix in another peered network. So, both VPC networks must have the different CIDR prefix.

My Organization [myproject] {ubuntu-test, default/10.146.0.2} <======> No organization [qwiklabs-gcp***]{privatenet-us-vm/172.16.0.2}

On No organization [qwiklabs-gcp***]: VPC network name is "privatenet".

VPC:
Firewall:


My Organization [myproject]: I used "default" VPC and default firewall.


Then, starting to create "VPC Network Peering".

- To create "VPC Network Peering" on myproject:
Networking => "VPC network" => "VPC network peering".
Click "Create Peering Connection".  name = "peering-to-lab".
Note: you must know Project ID and VPC network name for network destination.































It should show "Waiting for peer network to connect".

-  To create "VPC Network Peering" on another Project: On No organization [qwiklabs-gcp***], "Create Peering Connection".  name = "peering-to-mygcp".


After clicking "Create". It should show "Connected" on both projects (if configuration corrects) like.

On myproject:

- Finally, test connection: ssh to my vm and test (ssh) connection to {privatenet-us-vm/172.16.0.2}.
Note: (as firewall allow icmp/ssh). No need to do on firewall.

opun@ubuntu-test:~$ ssh 172.16.0.2
opun@172.16.0.2's password:
Linux privatenet-us-vm 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u4 (2018-08-21) x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Sep 19 06:22:05 2018 from 10.146.0.2
Could not chdir to home directory /home/opun: No such file or directory
$ w
 06:22:55 up 43 min,  2 users,  load average: 0.00, 0.00, 0.00
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
opun     pts/1    10.146.0.2       06:22    1.00s  0.00s  0.00s w ************
Reference: https://cloud.google.com/vpc/docs/vpc-peering