Intel IOMMU: Avoid memory allocation failures in dma map api calls

Intel IOMMU driver needs memory during DMA map calls to setup its internal
page tables and for other data structures.  As we all know that these DMA map
calls are mostly called in the interrupt context or with the spinlock held by
the upper level drivers(network/storage drivers), so in order to avoid any
memory allocation failure due to low memory issues, this patch makes memory
allocation by temporarily setting PF_MEMALLOC flags for the current task
before making memory allocation calls.

We evaluated mempools as a backup when kmem_cache_alloc() fails
and found that mempools are really not useful here because
 1) We don't know for sure how much to reserve in advance
 2) And mempools are not useful for GFP_ATOMIC case (as we call
    memory alloc functions with GFP_ATOMIC)

(akpm: point 2 is wrong...)

With PF_MEMALLOC flag set in the current->flags, the VM subsystem avoids any
watermark checks before allocating memory thus guarantee'ing the memory till
the last free page.  Further, looking at the code in mm/page_alloc.c in
__alloc_pages() function, looks like this flag is useful only in the
non-interrupt context.

If we are in the interrupt context and memory allocation in IOMMU driver fails
for some reason, then the DMA map api's will return failure and it is up to
the higher level drivers to retry.  Suppose, if upper level driver programs
the controller with the buggy DMA virtual address, the IOMMU will block that
DMA transaction when that happens thus preventing any corruption to main
memory.

So far in our test scenario, we were unable to create any memory allocation
failure inside dma map api calls.

Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Cc: Andi Kleen <ak@suse.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Muli Ben-Yehuda <muli@il.ibm.com>
Cc: "Siddha, Suresh B" <suresh.b.siddha@intel.com>
Cc: Arjan van de Ven <arjan@infradead.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: Greg KH <greg@kroah.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/drivers/pci/intel-iommu.c b/drivers/pci/intel-iommu.c
index 93ed771b3254b52a18127bf1cf58f964bbf1e50a..05630b44dbfc9b280ec5c3e5551849f6c784fae0 100644 (file)
--- a/drivers/pci/intel-iommu.c
+++ b/drivers/pci/intel-iommu.c
@@ -85,9 +85,31 @@ static struct kmem_cache *iommu_domain_cache;
 static struct kmem_cache *iommu_devinfo_cache;
 static struct kmem_cache *iommu_iova_cache;
 
+static inline void *iommu_kmem_cache_alloc(struct kmem_cache *cachep)
+{
+       unsigned int flags;
+       void *vaddr;
+
+       /* trying to avoid low memory issues */
+       flags = current->flags & PF_MEMALLOC;
+       current->flags |= PF_MEMALLOC;
+       vaddr = kmem_cache_alloc(cachep, GFP_ATOMIC);
+       current->flags &= (~PF_MEMALLOC | flags);
+       return vaddr;
+}
+
+
 static inline void *alloc_pgtable_page(void)
 {
-       return (void *)get_zeroed_page(GFP_ATOMIC);
+       unsigned int flags;
+       void *vaddr;
+
+       /* trying to avoid low memory issues */
+       flags = current->flags & PF_MEMALLOC;
+       current->flags |= PF_MEMALLOC;
+       vaddr = (void *)get_zeroed_page(GFP_ATOMIC);
+       current->flags &= (~PF_MEMALLOC | flags);
+       return vaddr;
 }
 
 static inline void free_pgtable_page(void *vaddr)
@@ -97,7 +119,7 @@ static inline void free_pgtable_page(void *vaddr)
 
 static inline void *alloc_domain_mem(void)
 {
-       return kmem_cache_alloc(iommu_domain_cache, GFP_ATOMIC);
+       return iommu_kmem_cache_alloc(iommu_domain_cache);
 }
 
 static inline void free_domain_mem(void *vaddr)
@@ -107,7 +129,7 @@ static inline void free_domain_mem(void *vaddr)
 
 static inline void * alloc_devinfo_mem(void)
 {
-       return kmem_cache_alloc(iommu_devinfo_cache, GFP_ATOMIC);
+       return iommu_kmem_cache_alloc(iommu_devinfo_cache);
 }
 
 static inline void free_devinfo_mem(void *vaddr)
@@ -117,7 +139,7 @@ static inline void free_devinfo_mem(void *vaddr)
 
 struct iova *alloc_iova_mem(void)
 {
-       return kmem_cache_alloc(iommu_iova_cache, GFP_ATOMIC);
+       return iommu_kmem_cache_alloc(iommu_iova_cache);
 }
 
 void free_iova_mem(struct iova *iova)