By Umair Mansoob | Edited by Mike Gangler

The Oracle Exadata Database Machine was first introduced by Oracle in 2008 and it has become one of the most popular database platforms to host Oracle databases. Exadata Machine is like a mini data center in itself, comprising of Database Servers, Storage Servers, InfiniBand Switches and Ethernet switch working together to deliver outstanding database performance.

Over the years, Exadata went through different deployment processes, as Oracle tries to make it simpler and error free. This article will take you through different phases of Exadata deployment and clearly define the responsibilities of each group involved, during each phase. At minimum, you should treat Exadata deployment as a small project and permanently dedicate resources for it.

Basically, Exadata deployment will involve many phases, and it can take up to three weeks before you can log in to Oracle Database running on your Exadata Machine. It starts by planning for target Exadata Machine, which includes” capacity planning, key deployment decisions, and licensing considerations.

Once you have narrow-downed your Exadata configurations, you can download and run Oracle Exadata Deployment Utility to generate different configuration files. OEDA utility will also provide the list of software and patches that you need to download from Metalink. After downloading and copying all the software to target Exadata Machine, you can apply Elastic Configuration and deploy Oracle software to Exadata Machine. Finally, verify all the software and hardware components once Exadata deployment is completed.

Planning

Exadata is a big investment, so it important that you plan Exadata deployment very carefully.  Following are some important considerations for your Exadata deployment.

Capacity Planning

You might be buying Exadata Machine for a particular application, or you may be planning to consolidate databases to Exadata platform. In any case, you should perform a detailed analysis of your existing database or databases, in order to do capacity planning for your Exadata Machine.

It is important to understand that Exadata offloads many processes to storage nodes, therefore requires less CPUs on compute nodes and also provides extraordinary compression ratio using Hybrid Columnar Compression, so it requires less storage. Capacity planning for Exadata is a little different from traditional hardware, but basics will still apply. You should gather information about CPU, IOPs and Physical Memory of an existing system and use them to size your Exadata Machine.

Deployment Decisions

Once you have sized your Exadata Machine, it is time for you to make important deployment decisions. Exadata deployment decisions can have lasting impacts on your hosted databases. For example, changing ASM disk group redundancy will require recreating that disk group causing application outage. Therefore, it is important to consider Exadata deployment decisions in detail before moving to the next phase of Exadata Deployment Life Cycle, including:

• ASM Redundancy Level,
• Virtualization,
• Network Isolation,
• Role Separation and
• Storage Type.

ASM redundancy is probably the most important Exadata deployment consideration since it will be hard to change it after the deployment. Though Oracle recommends using high redundancy level for Exadata Machine, you have an option to choose normal redundancy based on your requirements. If you are planning to deploy non-production databases, you might want to use normal redundancy to efficiently utilize expensive Exadata storage. It is also important to note that external ASM redundancy is not supported with Exadata systems.

As many of you already know, virtualization is supported with the release of X5 and it has become an important deployment consideration for many customers. I will not get into details in this article, but there are many benefits of using virtualization with Exadata Machine including license cost savings, security, and isolation. It is also important to understand that virtualizing Exadata Machine comes with administrative overhead.

Now you have multiple clusters to maintain instead of one. Network isolation can also be important if you have consolidated numbers of databases to Exadata Machine. You can achieve network isolation using two options: private VLAN and InfiniBand partitioning. You may also want role separation: You can deploy Exadata with Oracle user being a sole owner of GRID and RDBMS software, or you can have role separation by having two separate owners for GRID and RDBMS software. You can choose either way based on your organization standards.

Next, you need to decide what type of storage disk you want for your Exadata Machine. Exadata Machine comes with two storage disk options: Extreme Flash I/O and high capacity. As their names suggest, one is designed to provide more performance and better I/O response time, whereas the other is designed to provide you with higher storage capacity.

My recommendation will be to choose high capacity disks because new Exadata machines come with terabytes of flash cache and your application will be performing most of the read and write from flash cache. Similarly, you need to decide a backup location for your databases running on Exadata Machine. Exadata storage is not cheap, but it’s best to use an Oracle-supported NFS mount, like ZFS or any other Oracle-supported backup solution.

Licensing Considerations

When you buy Exadata Machine, you will also at least need to buy Oracle Database Enterprise Edition and Exadata Storage Software licenses. Given that Exadata compute nodes come with a higher number of CPUs, you are looking at significant investment to purchase Exadata Machine.

As a customer, you have two options to control Exadata software licensing cost:

1.  The Capacity on Demand (CoD) option to disable a subset of the cores on Exadata database servers to reduce licensing requirements. The maximum number of cores that can be disabled on an Exadata X5-2 Database Server is 60 percent.
2. Exadata now supports Oracle Virtual Machines (OVM). OVM can be used to create virtual machines with a specific number of cores to reduce licensing requirements, similar to the CoD option.

You can also use a virtual machine to save money on licensing since the options can be licensed on just the virtual machines using them instead of every physical core in the server. Additionally, if you are planning to use Oracle Advanced Compression or Oracle Advanced Security as an option and you don’t have existing licenses for it, consider buying them for the following reasons:

• You might be required to buy Oracle Advanced Security for compliance reasons, especially if your database contains credit card information and you need to encrypt data at rest and on the network.
• Though Exadata comes with its own free compression called Hybrid Columnar Compression (HCC), it does not support online transaction processing (OLTP) operations.

Oracle Advanced Compression can help you save on expensive storage and it can also improve performance by reducing I/O footprint.

ODEDA Utility

If you are one of the early adopters of Exadata Machine, you probably remember the old Exadata configuration excel sheet used for Exadata installation. A few years back, Oracle introduced a Java-based utility called Oracle Exadata Deployment Assistant (OEDA) in an effort to simplify the Exadata installation process.

You need to download the latest OEDA version and generate Exadata configuration files before the delivery of Exadata Machine. Once downloaded, work on the network and database administrators to fill out the following sections:

• IP addresses (Private & Public Network).
• Domain Name System (DNS).
• Network Time Protocol (NTP).
• Cluster name.
• Disk group name and redundancy.
• Email information for alerts.
• Oracle Configuration Manager (OCM).
• Auto Service Request (ASR).
• Grid Control Agent (OEM 12c).

If you are able to successfully run OEDA utility, it should generate the following configuration files for Exadata installation:

• OEDA XML file
• Network Check Script
• Installing Template
• Pre-configuration Files

Download Software

This deployment phase can be completed in parallel with any other phase, as long as you have run OEDA utility. It is important to understand that you will not be able to deploy Exadata software without downloading all the necessary software and patches from My Oracle Support. You can get a complete list of software and patches from Exadata configuration file generated by OEDA utility.

This phase consists of two parts: Download software and patches from My Oracle Support, then transfer them to Exadata Machine. You can download software and patches before you have possession of Exadata Machine, and once Exadata Machine is on your network, you can transfer them to your machine.

Here is an example of software and patches from the Exadata configuration file:

Elastic Configuration

By now, you should have the Oracle Exadata Database Machine shipped to your data center and it is ready for the next deployment phase. New Oracle Exadata machines are shipped with factory default IP addresses and these addresses are in the 172.16.2.1 to 172.16.7.254 range. Prior to connecting Oracle Exadata Database Machine to the network, ensure these IP addresses do not conflict with other addresses on the network

I strongly recommend running the checkip.sh script before connecting the network to avoid problems, even if a check was performed as part of the planning process before the machine was delivered. This is a very important step and you only need to run this step from one node only. This step will use your Exadata configuration file to assign new IP addresses and reboot all Exadata nodes including storage nodes.

Once this phase is complete, connect through SSH and run IBHOSTS to see if new IP addresses have been assigned. It is important that you don’t see any elastic keyword on the screen; otherwise, it shows that your elastic configuration is not complete.

Example of running elastic configuration:

./applyElasticConfig.sh -cf customer_name-configFile.xml

Now you can connect the Exadata Machine to your network.

Software Installation

During this phase, you install and configure Oracle software — including Oracle Grid and RDBMS on Exadata Machine. If you have ever installed Oracle Real Application Clusters in your organization, this phase will look very familiar to you. This phase is similar to installing Oracle Real Application Clusters on any other supported hardware configuration but more comprehensive.

This process was developed by Oracle engineers to simplify and automate the software installation process on Exadata Machine. This phase consists of 19 steps and you can run them one by one or you can run all of them together. I strongly recommend doing it one by one.

The following steps are performed by the Oracle Exadata OneCommand Utility and actual steps for a specific deployment may differ depending on your Exadata configuration. For example, you have to perform an extra step “Update Nodes for Eighth Rack” only for eighth rack deployment.

1. Validate Configuration File.
2. Update Nodes for Eighth Rack.
3. Create Virtual Machine.
4. Create Users.
5. Setup Cell Connectivity.
6. Create Cell Disks.
7. Create Grid Disks.
8. Configure Alerting.
9. Install Cluster Software.
10. Initialize Cluster Software.
11. Install Database Software.
12. Relink Database with RDS.
13. Create ASM Disk groups.
14. Create Databases.
15. Apply Security Fixes.
16. Install Exachk.
17. Setup ASR Alerting.
18. Create Installation Summary.
19. Re-secure Machine.

Once you complete all the 19 steps successfully, you have almost completed your Exadata deployment. Please make sure to change all the root passwords when logging in for the first time. All the operating system users are configured with default passwords.

Validate Installation

As mentioned earlier, your Exadata deployment is almost complete, but you should validate your Exadata installation before you start deploying any database on it. Exadata Machine is like a mini data center, containing many different types of hardware, including database servers, storage, and network switches. You should validate all the Exadata components including software and hardware

You can start by reviewing log files from Exadata software deployment phase; each step will generate a log file. Secondly, review Exachk and the installation summary report generated by Oracle Exadata OneCommand Utility for any reported errors. If you find any issues during this phase, it is important to get My Oracle Support involved to resolve these issues.

Conclusion

Oracle has invested a lot of time and effort to simplify the Exadata deployment process, but there are still many moving parts and hardware components. You might find the Exadata deployment process complex, especially if this is the first time you are deploying Exadata Machine in your organization. I hope this article will help you understand the Exadata deployment process better and make you feel comfortable enough to choose the Oracle Exadata Database Machine as your Oracle database platform. Good luck!