From: Srivatsa Vaddagiri <vatsa@in.ibm.com>
Date: Mon, 12 Dec 2005 08:37:07 +0000 (-0800)
Subject: [PATCH] Fix RCU race in access of nohz_cpu_mask
X-Git-Tag: v2.6.15-rc6~136
X-Git-Url: https://err.no/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=c3f5902325d3053986e7359f706581d8f032e72f;p=linux-2.6

[PATCH] Fix RCU race in access of nohz_cpu_mask

Accessing nohz_cpu_mask before incrementing rcp->cur is racy.  It can cause
tickless idle CPUs to be included in rsp->cpumask, which will extend
graceperiods unnecessarily.

Fix this race.  It has been tested using extensions to RCU torture module
that forces various CPUs to become idle.

Signed-off-by: Srivatsa Vaddagiri <vatsa@in.ibm.com>
Cc: Dipankar Sarma <dipankar@in.ibm.com>
Cc: "Paul E. McKenney" <paulmck@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
---

diff --git a/kernel/rcupdate.c b/kernel/rcupdate.c
index f45b91723d..48d3bce465 100644
--- a/kernel/rcupdate.c
+++ b/kernel/rcupdate.c
@@ -257,15 +257,23 @@ static void rcu_start_batch(struct rcu_ctrlblk *rcp, struct rcu_state *rsp,
 
 	if (rcp->next_pending &&
 			rcp->completed == rcp->cur) {
-		/* Can't change, since spin lock held. */
-		cpus_andnot(rsp->cpumask, cpu_online_map, nohz_cpu_mask);
-
 		rcp->next_pending = 0;
-		/* next_pending == 0 must be visible in __rcu_process_callbacks()
-		 * before it can see new value of cur.
+		/*
+		 * next_pending == 0 must be visible in
+		 * __rcu_process_callbacks() before it can see new value of cur.
 		 */
 		smp_wmb();
 		rcp->cur++;
+
+		/*
+		 * Accessing nohz_cpu_mask before incrementing rcp->cur needs a
+		 * Barrier  Otherwise it can cause tickless idle CPUs to be
+		 * included in rsp->cpumask, which will extend graceperiods
+		 * unnecessarily.
+		 */
+		smp_mb();
+		cpus_andnot(rsp->cpumask, cpu_online_map, nohz_cpu_mask);
+
 	}
 }