How do I allocate physically contiguous memory in Solaris?
Solaris provides several DDI/DKI routines prefaced with ddi_dma_ which abstract the characteristics of DMA, such that a single driver can be written to work across all Solaris supported platforms.
Specifically, ddi_dma_mem_alloc(9F) is the preferred routine to allocate driver memory suitable for DMA. The ddi_dma_attr(9S) structure controls the allocation of memory. Note that setting the dma_attr_sgllen to a length of 1 informs the DDI framework that physically contiguous memory is required. On Solaris Intel, each ddi_dma_cookie(9S) structure returned when the allocated memory is bound with the various ddi_dma*_bind(9F) contains the physical address of the memory, suitable for use in constructing scatter-gather lists. See the example isp(7D) device driver source, available for download for examples of using various ddi_dma_ prefixed routines.
NOTE: some versions of the DDI framework may not handle all rounding and size issues. In general, it's better to allocate the minimal number of contiguous pages required, in sizes falling on page boundaries.
Other allocators: on the Solaris OS on x86 Platforms, ddi_iopb_alloc(9F) by default allocates from a a very restricted pool of pages allocated at boot time. It's best not to use this allocator unless necessary (note that most modern adapters can perform DMA into the entire 32- and/or 64-bit address range). Also note that the kernel virtual memory allocated by kmem_alloc(9F) can be bound using ddi_dma_addr_bind_handle(9F), but this memory is not physically contiguous, and may appear above 32-bits on 64-bit or PAE enabled systems.
ddi_dma_mem_alloc(9F) is a recommended interface to use.
TIP: allocating very large blocks of physically contiguous memory:
The Solaris kernel memory model does not support large blocks of physically contiguous memory: as virtual/paged memory usage fragments the physical memory, there is no management performed to defragment physical memory. Also, various kernel data structures are sized dynamically at boot time, to match the system's memory size.
(One method to allocate a large physically contiguous memory block is to perform the allocation very early (when the driver is first attached), as memory resources are relatively unfragmented at this time. Note that kmem_alloc(9F) will typically also return memory which is physically contiguous early in system boot).
Source: http://developers.sun.com/solaris/developer/support/driver/faqs.html#NQ5.1




