memcgroup: fix zone isolation OOM

mem_cgroup_charge_common shows a tendency to OOM without good reason, when
a memhog goes well beyond its rss limit but with plenty of swap available.
Seen on x86 but not on PowerPC; seen when the next patch omits swapcache
from memcgroup, but we presume it can happen without.

mem_cgroup_isolate_pages is not quite satisfying reclaim's criteria for OOM
avoidance.  Already it has to scan beyond the nr_to_scan limit when it
finds a !LRU page or an active page when handling inactive or an inactive
page when handling active.  It needs to do exactly the same when it finds a
page from the wrong zone (the x86 tests had two zones, the PowerPC tests
had only one).

Don't increment scan and then decrement it in these cases, just move the
incrementation down.  Fix recent off-by-one when checking against
nr_to_scan.  Cut out "Check if the meta page went away from under us",
presumably left over from early debugging: no amount of such checks could
save us if this list really were being updated without locking.

This change does make the unlimited scan while holding two spinlocks
even worse - bad for latency and bad for containment; but that's a
separate issue which is better left to be fixed a little later.

Signed-off-by: Hugh Dickins <hugh@veritas.com>
Cc: Pavel Emelianov <xemul@openvz.org>
Acked-by: Balbir Singh <balbir@linux.vnet.ibm.com>
Cc: Paul Menage <menage@google.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Cc: Kirill Korotaev <dev@sw.ru>
Cc: Herbert Poetzl <herbert@13thfloor.at>
Cc: David Rientjes <rientjes@google.com>
Cc: Vaidyanathan Srinivasan <svaidy@linux.vnet.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index e8493fb2d69e0cec3a09b3ad4a00d0ff83145fbc..9793873d5a9035c5987c0e301298479e9b21a698 100644 (file)
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -260,24 +260,20 @@ unsigned long mem_cgroup_isolate_pages(unsigned long nr_to_scan,
        spin_lock(&mem_cont->lru_lock);
        scan = 0;
        list_for_each_entry_safe_reverse(pc, tmp, src, lru) {
-               if (scan++ > nr_to_scan)
+               if (scan >= nr_to_scan)
                        break;
                page = pc->page;
                VM_BUG_ON(!pc);
 
-               if (unlikely(!PageLRU(page))) {
-                       scan--;
+               if (unlikely(!PageLRU(page)))
                        continue;
-               }
 
                if (PageActive(page) && !active) {
                        __mem_cgroup_move_lists(pc, true);
-                       scan--;
                        continue;
                }
                if (!PageActive(page) && active) {
                        __mem_cgroup_move_lists(pc, false);
-                       scan--;
                        continue;
                }
 
@@ -288,13 +284,8 @@ unsigned long mem_cgroup_isolate_pages(unsigned long nr_to_scan,
                if (page_zone(page) != z)
                        continue;
 
-               /*
-                * Check if the meta page went away from under us
-                */
-               if (!list_empty(&pc->lru))
-                       list_move(&pc->lru, &pc_list);
-               else
-                       continue;
+               scan++;
+               list_move(&pc->lru, &pc_list);
 
                if (__isolate_lru_page(page, mode) == 0) {
                        list_move(&page->lru, dst);