Oracle 11gR2 Flash Cache Overview

What is it?

Flash Cache is an extension to the Oracle 11gR2 SGA that is stored on a high performance Flash storage device. It was originally introduced as a feature of the ExaData system but with 11gR2 it is available to all platforms. Benefits:
  • Flash cache memory is faster than disk storage
  • Flash cache memory is isolated from OS memory and 100% available for use by Oracle
  • Flash cache memory is cheaper than main memory
Drawbacks:
  • Flash cache memory is slower than main memory
  • Extra hardware and software configuration is required

Hardware Implementation

Flash Devices (also referred to as Solid State Disk or SSD) are available in multiple formats.

Form Factors

  • Hard drive replacement
    • Usually 3.5" format
    • Range from about 64gb ($150US) to 512gb(US$1500) sizes
    • Easy to install
  • Plug in cards
    • Size offerings vary by manufacturer
    • Includes direct interface to computer (usually PCI-e)
    • High performance
    • May be locked into certain vendor/driver choices depending on your server platform
  • USB Devices
    • Cheap and easily available
    • Hampered by USB interface
    • Does not have reliability or performance for permanent use

Performance

Type Access Time Throughput $/gb
Main Memory 5ns Multiple Gigs/sec $50-$150
Flash Hard Drive 0.1ms 250mb/sec SATA3 $2-$3/gb
Flash PCI Card ? 750mb-1000mb/sec -$5-$50/gb
Single Hard Drive 7200rpm 8ms 110mb/sec $0.10/gb
Storage Array Raid 10 (14 disks) 2ms 300mb/sec ~$15/gb
The hard drive replacements are relatively cheap and easy to use - a 3.5" 256gb SATA drive typically runs less than $1000 as of mid 2010. But keep in mind they are going to be limited by the SATA controller which may cause a bottleneck. SATA3 interfaces are typically saturated by SSD disks and run out of steam at about 250mb per second. Dedicated cards with flash memory are more expensive but faster. For example a SuperTalent 256mb PC card for x86 servers costs about $5000 for 256mb and can burst reads up to 1.4gb/second. Oracle's preferred option is - of course - their hardware offering. A fully loaded F10 PCIe Card (96gb) sells for about US$5,000 list price as of mid 2010.

Cost

Compare these prices with upgrades to main memory of typical DB servers: o a 96gb upgrade for a Dell 4U R910 Server currently costs about US$7200. o A 96gb upgrade to the main memory of a Sun Sparc M5000 which lists for over $15,000US without the required memory card which is at least an additional US$5,000.

Alternate Uses for Flash Devices

Most flash devices can also be used as regular disks. Since they are much faster than regular disks then it may make sense to hold frequently accessed files such as Temp datafiles. If you flash device has a battery back up you may even consider holding sensitive files such as Online Redo Logs or the Undo Tablespace. Always keep in mind the recoverability of the device you choose and make sure it works with your database recovery plan. After implementing sensitive files on a flash device I recommend

Implementation

It can be implemented using Oracle's F10 PICe Card in the ExaData, on Solaris, or on Oracle/Sun branded x86 Servers running Oracle Enterprise Linux. If you are running a system other than those listed above you can still use Flash Cache if you have a flash device installed on the system. Flash devices could be dedicated Flash disk replacements hooked up to a disk controller or even something as simple as a USB Flash drive. Note that the performance and reliability of a USB Flash Drive are not suitable for permanent or production use, but in a pinch can be used for a learning tool or lab session.

Software Implementation

Guy Harrison - using the 11gR2 Database Flash Cache - Guy tests out Flash Cache using a USB Key, not optimal but does show the concepts.

Usage

Once you have it set up this Flash Cache area can be used as a general extension to SGA or you can change the STORAGE clause of objects to make use of the Flash Cache. Keep in mind that if you are using Block Compression the blocks are compressed on disk and in memory. This makes the amount of data you can fit in the flash cache even higher since a compressed block may store more information than a non-compressed block. Guy Harrison - Flash Cache Keep Option - Some interesting pros and cons. Since this is the first release be careful what you 'Keep' as it may quickly fill your Flash Cache.

Monitoring

- Info on new metrics available to check performance of the Flash Cache.

References

Exadata_Smart_Flash_Cache_TWP_v5.pdf