Update on Hugepages Post
** UPDATED on Feb 14, 2014 **As an update on the hugepages post, I have researched the requirements for enabling the equivalent of Hugepages in AIX.
AIX uses the term largepages when referring to the closest equivalent of lInux hugepages. AIX large pages are 16mb in size, and are implemented differently than Linux hugepages. In fact, my research showed that of all operating systems I generally work with (Linux, Windows, AIX, and sun-sparc) Linux is the most difficult to implement.
Solaris will implement large pages automatically using ism (intimate shared memory). Nothing special needs to be done to do this.
Enabling Large Pages in AIX
In AIX, large pages are implemented as described below. In AIX and Solaris, as in Linux, largepages are not swappable and the full amount is allocated when the instance starts:
- Calculate the total size of all SGA’s you will need on the AIX server. This will be the SGA max size, since all memory will be allocated on startup. So if you have 3 instances, with SGA max size of 32G, 8G, and 4G, the total size is 44G. Add in a fudge factor of 2G to be safe, for a total of 46G.
Divide the total SGA in bytes by 16777216 (the AIX large page size) to get the total number of large page segments and add 1:
(46*1024*1024*1024)/(16777216)=2944+1=2945. So 2945 is the total number of large pages required.
Use vmo to allocate the largepages. You will need to allocate them to take effect immediately, and to persist after reboot, so the vmo command is run twice:
vmo –o lgpg_regions=2945 –o lgpg_size=16777216
vmo –r –o lgpg_regions=2945 –o lgpg_size=16777216
Grant the database installation owner (typically oracle) the required privileges to use large pages and to pin memory:
# Show pinshm settings:
vmo -o v_pinshm
v_pinshm = 0# set pinshm to 1 to allow shared memory to be pinned
vmo -o v_pinshm=1
vmo -r -o v_pinshm #persist change
chuser capabilities=CAP_BYPASS_RAC_VMM,CAP_PROPAGATE oracle
NOTE: after setting these, the oracle user will have to log out and back in for them to take effect.
In the Oracle database, set sga_max_size and sga_target to the same value, set lock_sga=true, and restart the instance eg:
Alter system set sga_max_size=16g scope=spfile sid=’*’;
Alter system set sga_target=16g scope=spfile sid=’*’;
Alter system set lock_sga=true scope=spfile sid=’*’;
To check the status of huge usage, run this command:
vmstat -P all
For 16M hugepages, look at the 16m row
System configuration: mem=61440MB
pgsz memory page
—– ————————– ————————————
siz avm fre re pi po fr sr cy
4K 5986176 1095680 61570 0 0 0 92 153 0
64K 35208 34938 270 0 0 0 0 0 0
16M 2241 2079 162 0 0 0 0 0 0
Addendum to Hugepages in Linux
Oracle has added a new ‘feature’ to OEL 6, called transparent hugepages. This is also in RHEL6 and SLES11 (Suse Linux). Oracle note 1557478.1 says that transparent hugepages should be disabled on servers running Oracle database. According to the note, the ‘feature’ can cause node reboots and performance problems. Transparent hugepages are allocated and deallocated like normal memory, so most likely the performance issues come from excessive CPU usage while managing the hugepages. In order to disable transparent hugepages, do the following:
Verify that transparent hugepages are enabled.
If the always is bracketed above, hugepages are enabled.
# grep /AnonHugePages /proc/meminfo
If this returns a value > 0kB, the kernel is uses Transparent HugePages
Preferred method to disable hugepages:
Add transparent_hugepage=never to /etc/grub.conf, and reboot the server, eg:
title Oracle Linux Server (2.6.32-300.25.1.el6uek.x86_64)
kernel /vmlinuz-2.6.32-300.25.1.el6uek.x86_64 ro root=LABEL=/ transparent_hugepage=never
Alternate method, disable in rc.local and reboot the server. Add the following section to rc.local and reboot:
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
Below is a link to spreadsheets containing common settings for hugepages (Linux) and largepages (AIX).