用普通内存模拟Persistent memory并测试Oracle 21c Persistent memory database
本站文章除注明转载外，均为本站原创： 转载自love wife love life —Roger的Oracle/MySQL/PostgreSQL数据恢复博客
本文链接地址: 用普通内存模拟Persistent memory并测试Oracle 21c Persistent memory database
Oracle 21c开始支持Persistent memory；由于没有Persistent memory device，我在想能否模拟一下呢。比如用我们的普通内存来模拟Persistent 内存。首先看下目前的虚拟机情况，我这里特意把虚拟机内存调整为6GB。
这里需要说明一点 的是，Linux 对于Persistent memory device的支持，是对内核版本有要求的，从查询的相关文档来看，应该是Linux 7.6+版本；即内核版本不低于4.1；我这里选择的是Oracle Linux 8.4，比较新。 另外可以通过下面的方式来确认一下，操作系统是否支持Persistent memory：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
[root@test ~]# egrep '(DAX|PMEM)' /boot/config-`uname -r` CONFIG_X86_PMEM_LEGACY_DEVICE=y CONFIG_X86_PMEM_LEGACY=m CONFIG_VIRTIO_PMEM=m CONFIG_BLK_DEV_PMEM=m CONFIG_NVDIMM_DAX=y CONFIG_DAX_DRIVER=y CONFIG_DAX=y CONFIG_DEV_DAX=m CONFIG_DEV_DAX_PMEM=m CONFIG_DEV_DAX_HMEM=m CONFIG_DEV_DAX_HMEM_DEVICES=y CONFIG_DEV_DAX_KMEM=m CONFIG_DEV_DAX_PMEM_COMPAT=m CONFIG_FS_DAX=y CONFIG_FS_DAX_PMD=y CONFIG_ARCH_HAS_PMEM_API=y |
从输出来看，我这里的虚拟机环境默认就支持了。另外可能还需要安装相关的包：
yum install librpmem librpmem-devel
yum install daxio
yum install ndctl libfabric
接下来开始进行相关配置调整：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
[root@test ~]# dmesg |grep BIOS [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x00000000000997ff] usable [ 0.000000] BIOS-e820: [mem 0x0000000000099800-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000dc000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bfedffff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bfee0000-0x00000000bfefefff] ACPI data [ 0.000000] BIOS-e820: [mem 0x00000000bfeff000-0x00000000bfefffff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x00000000bff00000-0x00000000bfffffff] usable [ 0.000000] BIOS-e820: [mem 0x00000000f0000000-0x00000000f7ffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec0ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffe0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000023fffffff] usable。 ---这里 [ 0.000000] SMBIOS 2.7 present. [ 0.000000] DMI: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020 [ 0.492521] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored |
从dmesg的信息输出来看，我这里可用的内存范围为4GB-8GB。 接下来修改一下grub引导文件。
|
1 2 3 4 5 6 7 8 9 |
[root@test ~]# cat /etc/default/grub GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="console" GRUB_CMDLINE_LINUX="crashkernel=auto resume=/dev/mapper/ol_192-swap rd.lvm.lv=ol_192/root rd.lvm.lv=ol_192/swap rhgb quiet memmap=4G!8G" GRUB_DISABLE_RECOVERY="true" GRUB_ENABLE_BLSCFG=true |
在grub文件中加入memmap=4G！8GB即可。 注意，这里要更新grub配置后，才重启操作系统，否则无法识别。
|
1 2 3 4 |
[root@test ~]# grub2-mkconfig -o /boot/grub2/grub.cfg Generating grub configuration file ... done [root@test ~]# reboot |
重启之后，我们来看下是否能看到Persistent device。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
[root@test ~]# dmesg |grep user [ 0.000000] user-defined physical RAM map: [ 0.000000] user: [mem 0x0000000000000000-0x00000000000997ff] usable [ 0.000000] user: [mem 0x0000000000099800-0x000000000009ffff] reserved [ 0.000000] user: [mem 0x00000000000dc000-0x00000000000fffff] reserved [ 0.000000] user: [mem 0x0000000000100000-0x00000000bfedffff] usable [ 0.000000] user: [mem 0x00000000bfee0000-0x00000000bfefefff] ACPI data [ 0.000000] user: [mem 0x00000000bfeff000-0x00000000bfefffff] ACPI NVS [ 0.000000] user: [mem 0x00000000bff00000-0x00000000bfffffff] usable [ 0.000000] user: [mem 0x00000000f0000000-0x00000000f7ffffff] reserved [ 0.000000] user: [mem 0x00000000fec00000-0x00000000fec0ffff] reserved [ 0.000000] user: [mem 0x00000000fee00000-0x00000000fee00fff] reserved [ 0.000000] user: [mem 0x00000000fffe0000-0x00000000ffffffff] reserved [ 0.000000] user: [mem 0x0000000100000000-0x00000001ffffffff] usable [ 0.000000] user: [mem 0x0000000200000000-0x000000023fffffff] persistent (type 12) [ 0.000000] user: [mem 0x0000000240000000-0x00000002ffffffff] persistent (type 12) [ 0.325031] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization [ 0.711123] thermal_sys: Registered thermal governor 'user_space' [root@test ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 9.3G 0 rom nvme0n1 259:0 0 40G 0 disk ├─nvme0n1p1 259:1 0 1G 0 part /boot └─nvme0n1p2 259:2 0 39G 0 part ├─ol_192-root 252:0 0 36.9G 0 lvm / └─ol_192-swap 252:1 0 2.1G 0 lvm [SWAP] nvme0n2 259:3 0 31G 0 disk └─nvme0n2p1 259:4 0 31G 0 part /dax pmem0 259:5 0 1G 0 disk pmem1 259:6 0 3G 0 disk |
这里可以发现，已经出现了pmem0/pmem1 disk了。也就是我们模拟的persistent device，将内存模拟成persistent memory device。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
[root@test ~]# ndctl list [ { "dev":"namespace1.0", "mode":"fsdax", "map":"mem", "size":3221225472, "sector_size":512, "blockdev":"pmem1" }, { "dev":"namespace0.0", "mode":"fsdax", "map":"mem", "size":1073741824, "sector_size":512, "blockdev":"pmem0" } ] |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
[root@test ~]# ndctl create-namespace -f -e namespace0.0 --mode=fsdax Warning: namespace0.0 unable to validate alignment { "dev":"namespace0.0", "mode":"fsdax", "map":"dev", "size":"1006.00 MiB (1054.87 MB)", "uuid":"2a7a43c9-9efd-46e0-9850-8d269243aa67", "sector_size":512, "align":2097152, "blockdev":"pmem0" } [root@test ~]# ndctl create-namespace -f -e namespace1.0 --mode=fsdax Warning: namespace1.0 unable to validate alignment { "dev":"namespace1.0", "mode":"fsdax", "map":"dev", "size":"2.95 GiB (3.17 GB)", "uuid":"70e9f2da-373f-4d9b-8552-2a9e1ef386ee", "sector_size":512, "align":4096, "blockdev":"pmem1" } |
接下来将上述2个盘进行格式化并挂载文件系统（注意只能是ext4或xfs才能支持dax）。
|
1 2 3 |
[root@test ~]# cat /proc/iomem |grep Persistent 200000000-23fffffff : Persistent Memory (legacy) 240000000-2ffffffff : Persistent Memory (legacy) |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
[root@test ~]# mkfs.ext4 -F /dev/pmem1 mke2fs 1.45.6 (20-Mar-2020) Creating filesystem with 773632 4k blocks and 193536 inodes Filesystem UUID: 042d2e55-bb1a-4fac-9d84-14c83fed2666 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912 Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done [root@test /]# mount -o dax,noatime /dev/pmem0 /mnt/dax/ [root@test /]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 3.3G 0 3.3G 0% /dev tmpfs 3.4G 624M 2.7G 19% /dev/shm tmpfs 3.4G 9.4M 3.3G 1% /run tmpfs 3.4G 0 3.4G 0% /sys/fs/cgroup /dev/mapper/ol_192-root 37G 24G 14G 64% / /dev/nvme0n1p1 1014M 322M 693M 32% /boot tmpfs 677M 1.2M 675M 1% /run/user/42 tmpfs 677M 0 677M 0% /run/user/0 /dev/pmem0 975M 2.5M 906M 1% /mnt/dax [root@test /]# mount |grep dax /dev/pmem0 on /mnt/dax type ext4 (rw,noatime,seclabel,dax=always) |
接下来创建Pmem filestore试试看;
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
[root@test ~]# su - oracle [oracle@test ~]$ pwd /home/oracle [oracle@test ~]$ sqlplus "/as sysdba" SQL*Plus: Release 21.0.0.0.0 - Production on Mon Aug 23 14:20:12 2021 Version 21.3.0.0.0 Copyright (c) 1982, 2021, Oracle. All rights reserved. Connected to an idle instance. SQL> startup nomount ORACLE instance started. Total System Global Area 1140848688 bytes Fixed Size 9685040 bytes Variable Size 738197504 bytes Database Buffers 385875968 bytes Redo Buffers 7090176 bytes SQL> CREATE PMEM FILESTORE db_pmemfs 2 MOUNTPOINT '/mnt/dax/21c_pmemfs' 3 BACKINGFILE '/mnt/dax/21c' 4 SIZE 200M 5 BLOCKSIZE 8K 6 AUTOEXTEND ON NEXT 10M 7 MAXSIZE 800M; CREATE PMEM FILESTORE db_pmemfs * ERROR at line 1: ORA-12755: Feature PMEM FILESTORE is disabled due to unsupported capability. |
看来对于该功能的使用，Oracle做了相关的限制，通常来讲是通过隐含参数控制，不过这里我调整了如下2个参数，发现均不行：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
Enter value for par: _exadata_feature_on old 3: WHERE x.indx = y.indx AND x.ksppinm LIKE '%&par%' new 3: WHERE x.indx = y.indx AND x.ksppinm LIKE '%_exadata_feature_on%' NAME VALUE DESCRIB ------------------------------ -------------------- ------------------------------------------------------------ _exadata_feature_on TRUE Exadata Feature On SQL> / Enter value for par: _simulate_dax_storage old 3: WHERE x.indx = y.indx AND x.ksppinm LIKE '%&par%' new 3: WHERE x.indx = y.indx AND x.ksppinm LIKE '%_simulate_dax_storage%' NAME VALUE DESCRIB ------------------------------ -------------------- ------------------------------------------------------------ _simulate_dax_storage ALL Simulate log on DAX storage |
从Oracle Mos文档来看，确实有关于Pmemsistent memory相关的bug，而且都出现在oracle exadata上，由此可见Oracle Exadata是支持该功能的。
Bug 29648928 – pga heap rather than ipc heap used for coalesced writes to exadata pmemlog (Doc ID 29648928.8)
Bug 31331038 – ORA-600 error in ADG SYNC mode on Exadata with PMEMlog (Doc ID 31331038.8)
Oracle Exadata Database Machine Setup/Configuration Best Practices (Doc ID 1274318.1)
抽空再研究研究该问题；感觉还是蛮有意思的；从Oracle官方的测试数据来看，Persistent memory设备的性能比NVME SSD的性能要高10-20倍。
Leave a Reply
You must be logged in to post a comment.