From: Roland Dreier Date: Wed, 10 Oct 2007 02:59:18 +0000 (-0700) Subject: mlx4_core: Use mmiowb() to avoid firmware commands getting jumbled up X-Git-Tag: v2.6.24-rc1~1453^2~28 X-Git-Url: https://err.no/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=2e61c646edfa013203e3428762f8d6a72e10bdea;p=linux-2.6 mlx4_core: Use mmiowb() to avoid firmware commands getting jumbled up Firmware commands are sent to the HCA by writing multiple words to a command register block. Access to this block of registers is serialized with a mutex. However, on large SGI systems writes to the register block may be reordered within the system interconnect and reach the HCA in a different order than they were issued (even with the mutex). Fix this by adding an mmiowb() before dropping the mutex. This bug was observed with real workloads with the similar FW command code in the mthca driver, and adding the mmiowb() as in commit 66547550 ("IB/mthca: Use mmiowb() to avoid firmware commands getting jumbled up") was confirmed to fix the problems, so we should add the same fix to mlx4. Signed-off-by: Roland Dreier --- diff --git a/drivers/net/mlx4/cmd.c b/drivers/net/mlx4/cmd.c index b540820e9c..db49051b97 100644 --- a/drivers/net/mlx4/cmd.c +++ b/drivers/net/mlx4/cmd.c @@ -184,6 +184,13 @@ static int mlx4_cmd_post(struct mlx4_dev *dev, u64 in_param, u64 out_param, (event ? (1 << HCR_E_BIT) : 0) | (op_modifier << HCR_OPMOD_SHIFT) | op), hcr + 6); + + /* + * Make sure that our HCR writes don't get mixed in with + * writes from another CPU starting a FW command. + */ + mmiowb(); + cmd->toggle = cmd->toggle ^ 1; ret = 0;