戰(zhàn),解決訂單超賣并發(fā)問題)
做電商后端開發(fā)商品超賣應(yīng)該算是老生常談的并發(fā)難題了。之前我參與過一個(gè)秒殺小活動(dòng)庫存設(shè)置 100 件最后下單成功 130 多單直接出現(xiàn)超賣后續(xù)還要人工處理退款給業(yè)務(wù)造成不小麻煩。一開始我們用數(shù)據(jù)庫樂觀鎖去處理并發(fā)量不高的時(shí)候還好一旦請(qǐng)求量上來大量請(qǐng)求直接打在數(shù)據(jù)庫數(shù)據(jù)庫 CPU 飆升接口響應(yīng)直接超時(shí)。后來調(diào)研選型最終采用 Redisson 分布式鎖來解決該問題。今天結(jié)合線上踩過的坑把 Redisson 分布式鎖落地完整過程記錄下來。一、為什么不能直接用原生 Redis 實(shí)現(xiàn)分布式鎖很多同學(xué)剛開始會(huì)直接用SETNX手寫 Redis 鎖。寫起來看著簡單但是坑非常多。比如鎖超時(shí)、業(yè)務(wù)還沒執(zhí)行完鎖就釋放鎖被其他線程誤釋放主從切換鎖丟失等一堆問題。如果自己全部去處理這些邊界代碼會(huì)寫的非常復(fù)雜稍有遺漏就會(huì)出線上故障。而 Redisson 是 Redis 官方推薦的 Java 客戶端它把鎖續(xù)期、鎖可重入、鎖釋放、主從等各種復(fù)雜邏輯全部封裝好了我們直接調(diào)用 API 即可生產(chǎn)環(huán)境穩(wěn)定性要高很多。二、項(xiàng)目引入 Redisson 依賴SpringBoot 項(xiàng)目pom.xml 引入依賴注意版本要和你的 SpringBoot 版本兼容。dependency groupIdorg.redisson/groupId artifactIdredisson?spring?boot?starter/artifactId version3.23.5/version /dependencyapplication.yml 配置 Redis 連接信息spring: redis: host: 127.0.0.1 port: 6379 password: database: 0三、訂單扣庫存超賣錯(cuò)誤示例問題代碼先看一段有并發(fā)問題的代碼這也是很多新手容易寫出的業(yè)務(wù)邏輯。Service public class OrderServiceImpl implements OrderService { Autowired private GoodsMapper goodsMapper; Autowired private OrderMapper orderMapper; Override Transactional public void createOrder(Long goodsId) { //1.查詢庫存 Goods goods goodsMapper.selectById(goodsId); if(goods.getStock() 0){ throw new RuntimeException(商品庫存不足); } //2.扣減庫存 goods.setStock(goods.getStock() -1); goodsMapper.updateById(goods); //3.生成訂單 Order order new Order(); order.setGoodsId(goodsId); orderMapper.insert(order); } }問題高并發(fā)場景多個(gè)線程同時(shí)讀到同一個(gè)庫存數(shù)值全部通過庫存判斷就會(huì)產(chǎn)生超賣。單純數(shù)據(jù)庫事務(wù)解決不了這個(gè)問題事務(wù)只能保證原子性不能擋住并發(fā)讀取。四、Redisson 分布式鎖改造后的業(yè)務(wù)代碼這里使用 Redisson 可重入鎖RLock直接上鎖之后再執(zhí)行庫存判斷和下單邏輯。Service public class OrderServiceImpl implements OrderService { Autowired private GoodsMapper goodsMapper; Autowired private OrderMapper orderMapper; Autowired private RedissonClient redissonClient; Override Transactional public void createOrder(Long goodsId) { //定義鎖key同一個(gè)商品用同一個(gè)key String lockKey lock:goods: goodsId; RLock rLock redissonClient.getLock(lockKey); try { //嘗試獲取鎖等待3秒鎖自動(dòng)釋放時(shí)間30秒 boolean getLock rLock.tryLock(3,30, TimeUnit.SECONDS); if(!getLock){ throw new RuntimeException(請(qǐng)求繁忙請(qǐng)稍后重試); } //獲取鎖成功執(zhí)行業(yè)務(wù) Goods goods goodsMapper.selectById(goodsId); if(goods.getStock() 0){ throw new RuntimeException(商品庫存不足); } goods.setStock(goods.getStock() -1); goodsMapper.updateById(goods); Order order new Order(); order.setGoodsId(goodsId); orderMapper.insert(order); } catch (InterruptedException e) { Thread.currentThread().interrupt(); throw new RuntimeException(獲取鎖異常); } finally { //必須判斷當(dāng)前線程持有鎖才釋放防止誤釋放別人的鎖 if(rLock.isHeldByCurrentThread()){ rLock.unlock(); } } } }這里有個(gè)細(xì)節(jié)finally釋放鎖的時(shí)候一定要判斷isHeldByCurrentThread()。如果業(yè)務(wù)執(zhí)行異常鎖已經(jīng)過期釋放直接調(diào)用 unlock會(huì)把其他線程拿到的鎖給釋放掉這個(gè)坑不少人踩過。Redisson 自帶的看門狗機(jī)制如果業(yè)務(wù)執(zhí)行時(shí)間超過設(shè)置的 30 秒會(huì)自動(dòng)給鎖續(xù)期避免業(yè)務(wù)沒跑完鎖提前釋放這也是手寫 SETNX 鎖很難實(shí)現(xiàn)的能力。五、實(shí)際生產(chǎn)落地需要注意的幾個(gè)坑鎖粒度不要太大不要直接寫一個(gè)全局鎖所有商品共用同一個(gè) key。不同商品使用不同的 lockKey例如lock:goods:{goodsId}并發(fā)性能會(huì)高很多。分布式鎖不能完全替代數(shù)據(jù)庫層面防護(hù)分布式鎖也不是萬無一失。Redis 主從切換鎖有可能丟失。建議數(shù)據(jù)庫層面加上庫存版本號(hào)樂觀鎖作為兜底雙重防護(hù)杜絕超賣。簡單樂觀鎖更新示例update goods set stock stock?1, version version1 where id #{goodsId} and stock0 and version #{version};鎖和事務(wù)順序問題鎖要加在事務(wù)外面。如果鎖在事務(wù)內(nèi)部事務(wù)還沒提交完成鎖就釋放其他線程讀到未提交的數(shù)據(jù)依舊會(huì)出現(xiàn)并發(fā)問題。區(qū)分鎖類型普通搶購場景用可重入鎖 RLock讀寫場景可以考慮 Redisson 讀寫鎖如果是集群環(huán)境要配置 Redisson 的 Redis 集群模式不要單機(jī)配置。六、總結(jié)超賣問題本質(zhì)就是并發(fā)下共享資源競爭。數(shù)據(jù)庫樂觀鎖適合并發(fā)壓力一般的場景當(dāng) QPS 較高大量請(qǐng)求沖擊數(shù)據(jù)庫就需要引入分布式鎖。Redisson 把分布式鎖里面各種棘手問題鎖續(xù)期、可重入、防止誤釋放、異常處理全部封裝好了開發(fā)不用重復(fù)造輪子。但千萬不要覺得上了分布式鎖就萬事大吉。Redis 畢竟是中間件存在網(wǎng)絡(luò)、主從切換等不可控因素。生產(chǎn)環(huán)境建議Redisson 分布式鎖 數(shù)據(jù)庫樂觀鎖雙重兜底最大程度避免超賣事故。實(shí)際項(xiàng)目里面還可以繼續(xù)優(yōu)化結(jié)合 MQ 異步削峰把下單邏輯異步化進(jìn)一步提升系統(tǒng)承載能力。Csdn.otftcf.cN/Article/details/23979.sHtMLCsdn.otftcf.cN/Article/details/01851.sHtMLCsdn.otftcf.cN/Article/details/86259.sHtMLCsdn.otftcf.cN/Article/details/45039.sHtMLCsdn.otftcf.cN/Article/details/03930.sHtMLCsdn.otftcf.cN/Article/details/96005.sHtMLCsdn.otftcf.cN/Article/details/09542.sHtMLCsdn.otftcf.cN/Article/details/03508.sHtMLCsdn.otftcf.cN/Article/details/68259.sHtMLCsdn.otftcf.cN/Article/details/41700.sHtMLCsdn.otftcf.cN/Article/details/43631.sHtMLCsdn.otftcf.cN/Article/details/26368.sHtMLCsdn.otftcf.cN/Article/details/53243.sHtMLCsdn.otftcf.cN/Article/details/29143.sHtMLCsdn.otftcf.cN/Article/details/52681.sHtMLCsdn.otftcf.cN/Article/details/37799.sHtMLCsdn.otftcf.cN/Article/details/32842.sHtMLCsdn.otftcf.cN/Article/details/46487.sHtMLCsdn.otftcf.cN/Article/details/31573.sHtMLCsdn.otftcf.cN/Article/details/26301.sHtMLCsdn.otftcf.cN/Article/details/33388.sHtMLCsdn.otftcf.cN/Article/details/98462.sHtMLCsdn.otftcf.cN/Article/details/14709.sHtMLCsdn.otftcf.cN/Article/details/31135.sHtMLCsdn.otftcf.cN/Article/details/09590.sHtMLCsdn.otftcf.cN/Article/details/28372.sHtMLCsdn.otftcf.cN/Article/details/08326.sHtMLCsdn.otftcf.cN/Article/details/29296.sHtMLCsdn.otftcf.cN/Article/details/47713.sHtMLCsdn.otftcf.cN/Article/details/92396.sHtMLCsdn.otftcf.cN/Article/details/86993.sHtMLCsdn.otftcf.cN/Article/details/22430.sHtMLCsdn.otftcf.cN/Article/details/49670.sHtMLCsdn.otftcf.cN/Article/details/80132.sHtMLCsdn.otftcf.cN/Article/details/70169.sHtMLCsdn.otftcf.cN/Article/details/42557.sHtMLCsdn.otftcf.cN/Article/details/26626.sHtMLCsdn.otftcf.cN/Article/details/62196.sHtMLCsdn.otftcf.cN/Article/details/64315.sHtMLCsdn.otftcf.cN/Article/details/12170.sHtMLCsdn.otftcf.cN/Article/details/16667.sHtMLCsdn.otftcf.cN/Article/details/52431.sHtMLCsdn.otftcf.cN/Article/details/73167.sHtMLCsdn.otftcf.cN/Article/details/44001.sHtMLCsdn.otftcf.cN/Article/details/32053.sHtMLCsdn.otftcf.cN/Article/details/87683.sHtMLCsdn.otftcf.cN/Article/details/96883.sHtMLCsdn.otftcf.cN/Article/details/78010.sHtMLCsdn.otftcf.cN/Article/details/77173.sHtMLCsdn.otftcf.cN/Article/details/81350.sHtMLCsdn.otftcf.cN/Article/details/91726.sHtMLCsdn.otftcf.cN/Article/details/10117.sHtMLCsdn.otftcf.cN/Article/details/49112.sHtMLCsdn.otftcf.cN/Article/details/73668.sHtMLCsdn.otftcf.cN/Article/details/35555.sHtMLCsdn.otftcf.cN/Article/details/84266.sHtMLCsdn.otftcf.cN/Article/details/18159.sHtMLCsdn.otftcf.cN/Article/details/53697.sHtMLCsdn.otftcf.cN/Article/details/41263.sHtMLCsdn.otftcf.cN/Article/details/02516.sHtMLCsdn.otftcf.cN/Article/details/81008.sHtMLCsdn.otftcf.cN/Article/details/31484.sHtMLCsdn.otftcf.cN/Article/details/80416.sHtMLCsdn.otftcf.cN/Article/details/48781.sHtMLCsdn.otftcf.cN/Article/details/54090.sHtMLCsdn.otftcf.cN/Article/details/07634.sHtMLCsdn.otftcf.cN/Article/details/02660.sHtMLCsdn.otftcf.cN/Article/details/45964.sHtMLCsdn.otftcf.cN/Article/details/72954.sHtMLCsdn.otftcf.cN/Article/details/49932.sHtMLCsdn.otftcf.cN/Article/details/38109.sHtMLCsdn.otftcf.cN/Article/details/34930.sHtMLCsdn.otftcf.cN/Article/details/16979.sHtMLCsdn.otftcf.cN/Article/details/08181.sHtMLCsdn.otftcf.cN/Article/details/81858.sHtMLCsdn.otftcf.cN/Article/details/10695.sHtMLCsdn.otftcf.cN/Article/details/04382.sHtMLCsdn.otftcf.cN/Article/details/68399.sHtMLCsdn.otftcf.cN/Article/details/03077.sHtMLCsdn.otftcf.cN/Article/details/87942.sHtMLCsdn.otftcf.cN/Article/details/48596.sHtMLCsdn.otftcf.cN/Article/details/35596.sHtMLCsdn.otftcf.cN/Article/details/87671.sHtMLCsdn.otftcf.cN/Article/details/32704.sHtMLCsdn.otftcf.cN/Article/details/56154.sHtMLCsdn.otftcf.cN/Article/details/64787.sHtMLCsdn.otftcf.cN/Article/details/80695.sHtMLCsdn.otftcf.cN/Article/details/66814.sHtMLCsdn.otftcf.cN/Article/details/96017.sHtMLCsdn.otftcf.cN/Article/details/97934.sHtMLCsdn.otftcf.cN/Article/details/40554.sHtMLCsdn.otftcf.cN/Article/details/13217.sHtMLCsdn.otftcf.cN/Article/details/39354.sHtMLCsdn.otftcf.cN/Article/details/05694.sHtMLCsdn.otftcf.cN/Article/details/54209.sHtMLCsdn.otftcf.cN/Article/details/97485.sHtMLCsdn.otftcf.cN/Article/details/27044.sHtMLCsdn.otftcf.cN/Article/details/78947.sHtMLCsdn.otftcf.cN/Article/details/31397.sHtMLCsdn.otftcf.cN/Article/details/51508.sHtMLCsdn.otftcf.cN/Article/details/95487.sHtMLCsdn.otftcf.cN/Article/details/52507.sHtMLCsdn.otftcf.cN/Article/details/17509.sHtMLCsdn.otftcf.cN/Article/details/55825.sHtMLCsdn.otftcf.cN/Article/details/72473.sHtMLCsdn.otftcf.cN/Article/details/01597.sHtMLCsdn.otftcf.cN/Article/details/21816.sHtMLCsdn.otftcf.cN/Article/details/96053.sHtMLCsdn.otftcf.cN/Article/details/80368.sHtMLCsdn.otftcf.cN/Article/details/40856.sHtMLCsdn.otftcf.cN/Article/details/96896.sHtMLCsdn.otftcf.cN/Article/details/89327.sHtMLCsdn.otftcf.cN/Article/details/71057.sHtMLCsdn.otftcf.cN/Article/details/49181.sHtMLCsdn.otftcf.cN/Article/details/10540.sHtMLCsdn.otftcf.cN/Article/details/94310.sHtMLCsdn.otftcf.cN/Article/details/19718.sHtMLCsdn.otftcf.cN/Article/details/48328.sHtMLCsdn.otftcf.cN/Article/details/52948.sHtMLCsdn.otftcf.cN/Article/details/17849.sHtMLCsdn.otftcf.cN/Article/details/87762.sHtMLCsdn.otftcf.cN/Article/details/67646.sHtMLCsdn.otftcf.cN/Article/details/12087.sHtMLCsdn.otftcf.cN/Article/details/45653.sHtMLCsdn.otftcf.cN/Article/details/50463.sHtMLCsdn.otftcf.cN/Article/details/72458.sHtMLCsdn.otftcf.cN/Article/details/23479.sHtMLCsdn.otftcf.cN/Article/details/28745.sHtMLCsdn.otftcf.cN/Article/details/23736.sHtMLCsdn.otftcf.cN/Article/details/13037.sHtMLCsdn.otftcf.cN/Article/details/07815.sHtMLCsdn.otftcf.cN/Article/details/16100.sHtMLCsdn.otftcf.cN/Article/details/11657.sHtMLCsdn.otftcf.cN/Article/details/59554.sHtMLCsdn.otftcf.cN/Article/details/75620.sHtMLCsdn.otftcf.cN/Article/details/98442.sHtMLCsdn.otftcf.cN/Article/details/72996.sHtMLCsdn.otftcf.cN/Article/details/54153.sHtMLCsdn.otftcf.cN/Article/details/05563.sHtMLCsdn.otftcf.cN/Article/details/32834.sHtMLCsdn.otftcf.cN/Article/details/06743.sHtMLCsdn.otftcf.cN/Article/details/50318.sHtMLCsdn.otftcf.cN/Article/details/39199.sHtMLCsdn.otftcf.cN/Article/details/06334.sHtMLCsdn.otftcf.cN/Article/details/70916.sHtMLCsdn.otftcf.cN/Article/details/01456.sHtMLCsdn.otftcf.cN/Article/details/22133.sHtMLCsdn.otftcf.cN/Article/details/21355.sHtMLCsdn.otftcf.cN/Article/details/65337.sHtMLCsdn.otftcf.cN/Article/details/08874.sHtMLCsdn.otftcf.cN/Article/details/15346.sHtMLCsdn.otftcf.cN/Article/details/61001.sHtMLCsdn.otftcf.cN/Article/details/83231.sHtMLCsdn.otftcf.cN/Article/details/35410.sHtMLCsdn.otftcf.cN/Article/details/38547.sHtMLCsdn.otftcf.cN/Article/details/13730.sHtMLCsdn.otftcf.cN/Article/details/18038.sHtMLCsdn.otftcf.cN/Article/details/07964.sHtMLCsdn.otftcf.cN/Article/details/68073.sHtMLCsdn.otftcf.cN/Article/details/97380.sHtMLCsdn.otftcf.cN/Article/details/10373.sHtMLCsdn.otftcf.cN/Article/details/07387.sHtMLCsdn.otftcf.cN/Article/details/11740.sHtMLCsdn.otftcf.cN/Article/details/44462.sHtMLCsdn.otftcf.cN/Article/details/17056.sHtMLCsdn.otftcf.cN/Article/details/99535.sHtMLCsdn.otftcf.cN/Article/details/38460.sHtMLCsdn.otftcf.cN/Article/details/69853.sHtMLCsdn.otftcf.cN/Article/details/95709.sHtMLCsdn.otftcf.cN/Article/details/62756.sHtMLCsdn.otftcf.cN/Article/details/69651.sHtMLCsdn.otftcf.cN/Article/details/45152.sHtMLCsdn.otftcf.cN/Article/details/88076.sHtMLCsdn.otftcf.cN/Article/details/08307.sHtMLCsdn.otftcf.cN/Article/details/28729.sHtMLCsdn.otftcf.cN/Article/details/31845.sHtMLCsdn.otftcf.cN/Article/details/82680.sHtMLCsdn.otftcf.cN/Article/details/85639.sHtMLCsdn.otftcf.cN/Article/details/20795.sHtMLCsdn.otftcf.cN/Article/details/60202.sHtMLCsdn.otftcf.cN/Article/details/32287.sHtMLCsdn.otftcf.cN/Article/details/27956.sHtMLCsdn.otftcf.cN/Article/details/43084.sHtMLCsdn.otftcf.cN/Article/details/17386.sHtMLCsdn.otftcf.cN/Article/details/24710.sHtMLCsdn.otftcf.cN/Article/details/07056.sHtMLCsdn.otftcf.cN/Article/details/41540.sHtMLCsdn.otftcf.cN/Article/details/47551.sHtMLCsdn.otftcf.cN/Article/details/14605.sHtMLCsdn.otftcf.cN/Article/details/02323.sHtMLCsdn.otftcf.cN/Article/details/68584.sHtMLCsdn.otftcf.cN/Article/details/19625.sHtMLCsdn.otftcf.cN/Article/details/92536.sHtMLCsdn.otftcf.cN/Article/details/40090.sHtMLCsdn.otftcf.cN/Article/details/06626.sHtMLCsdn.otftcf.cN/Article/details/86829.sHtMLCsdn.otftcf.cN/Article/details/95896.sHtMLCsdn.otftcf.cN/Article/details/21481.sHtMLCsdn.otftcf.cN/Article/details/79061.sHtMLCsdn.otftcf.cN/Article/details/87142.sHtMLCsdn.otftcf.cN/Article/details/49016.sHtMLCsdn.otftcf.cN/Article/details/88651.sHtMLCsdn.otftcf.cN/Article/details/44621.sHtMLCsdn.otftcf.cN/Article/details/61641.sHtMLCsdn.otftcf.cN/Article/details/81629.sHtMLCsdn.otftcf.cN/Article/details/28469.sHtMLCsdn.otftcf.cN/Article/details/08291.sHtMLCsdn.otftcf.cN/Article/details/83635.sHtMLCsdn.otftcf.cN/Article/details/16256.sHtMLCsdn.otftcf.cN/Article/details/05819.sHtMLCsdn.otftcf.cN/Article/details/34924.sHtMLCsdn.otftcf.cN/Article/details/70613.sHtMLCsdn.otftcf.cN/Article/details/03429.sHtMLCsdn.otftcf.cN/Article/details/14900.sHtMLCsdn.otftcf.cN/Article/details/56319.sHtMLCsdn.otftcf.cN/Article/details/42601.sHtMLCsdn.otftcf.cN/Article/details/97066.sHtMLCsdn.otftcf.cN/Article/details/78208.sHtMLCsdn.otftcf.cN/Article/details/46005.sHtMLCsdn.otftcf.cN/Article/details/13604.sHtMLCsdn.otftcf.cN/Article/details/25692.sHtMLCsdn.otftcf.cN/Article/details/01682.sHtMLCsdn.otftcf.cN/Article/details/71608.sHtMLCsdn.otftcf.cN/Article/details/62713.sHtMLCsdn.otftcf.cN/Article/details/96572.sHtMLCsdn.otftcf.cN/Article/details/68100.sHtMLCsdn.otftcf.cN/Article/details/24930.sHtMLCsdn.otftcf.cN/Article/details/71285.sHtMLCsdn.otftcf.cN/Article/details/45304.sHtMLCsdn.otftcf.cN/Article/details/47398.sHtMLCsdn.otftcf.cN/Article/details/98880.sHtMLCsdn.otftcf.cN/Article/details/04272.sHtMLCsdn.otftcf.cN/Article/details/40180.sHtMLCsdn.otftcf.cN/Article/details/92898.sHtMLCsdn.otftcf.cN/Article/details/22485.sHtMLCsdn.otftcf.cN/Article/details/15623.sHtMLCsdn.otftcf.cN/Article/details/00005.sHtMLCsdn.otftcf.cN/Article/details/93926.sHtMLCsdn.otftcf.cN/Article/details/40654.sHtMLCsdn.otftcf.cN/Article/details/25135.sHtMLCsdn.otftcf.cN/Article/details/22546.sHtMLCsdn.otftcf.cN/Article/details/55708.sHtMLCsdn.otftcf.cN/Article/details/20531.sHtMLCsdn.otftcf.cN/Article/details/20212.sHtMLCsdn.otftcf.cN/Article/details/01523.sHtMLCsdn.otftcf.cN/Article/details/92810.sHtMLCsdn.otftcf.cN/Article/details/95947.sHtMLCsdn.otftcf.cN/Article/details/36445.sHtMLCsdn.otftcf.cN/Article/details/42454.sHtMLCsdn.otftcf.cN/Article/details/70617.sHtMLCsdn.otftcf.cN/Article/details/14485.sHtMLCsdn.otftcf.cN/Article/details/85106.sHtMLCsdn.otftcf.cN/Article/details/64502.sHtMLCsdn.otftcf.cN/Article/details/89422.sHtMLCsdn.otftcf.cN/Article/details/15062.sHtMLCsdn.otftcf.cN/Article/details/88862.sHtMLCsdn.otftcf.cN/Article/details/22217.sHtMLCsdn.otftcf.cN/Article/details/36578.sHtMLCsdn.otftcf.cN/Article/details/27542.sHtMLCsdn.otftcf.cN/Article/details/52888.sHtMLCsdn.otftcf.cN/Article/details/81389.sHtMLCsdn.otftcf.cN/Article/details/06018.sHtMLCsdn.otftcf.cN/Article/details/17286.sHtMLCsdn.otftcf.cN/Article/details/46041.sHtMLCsdn.otftcf.cN/Article/details/58331.sHtMLCsdn.otftcf.cN/Article/details/43907.sHtMLCsdn.otftcf.cN/Article/details/24533.sHtMLCsdn.otftcf.cN/Article/details/71350.sHtMLCsdn.otftcf.cN/Article/details/16291.sHtMLCsdn.otftcf.cN/Article/details/87237.sHtMLCsdn.otftcf.cN/Article/details/70559.sHtMLCsdn.otftcf.cN/Article/details/08486.sHtMLCsdn.otftcf.cN/Article/details/57979.sHtMLCsdn.otftcf.cN/Article/details/86246.sHtMLCsdn.otftcf.cN/Article/details/83094.sHtMLCsdn.otftcf.cN/Article/details/51934.sHtMLCsdn.otftcf.cN/Article/details/28453.sHtMLCsdn.otftcf.cN/Article/details/63835.sHtMLCsdn.otftcf.cN/Article/details/39699.sHtMLCsdn.otftcf.cN/Article/details/72840.sHtMLCsdn.otftcf.cN/Article/details/03446.sHtMLCsdn.otftcf.cN/Article/details/61223.sHtMLCsdn.otftcf.cN/Article/details/83903.sHtMLCsdn.otftcf.cN/Article/details/61242.sHtMLCsdn.otftcf.cN/Article/details/60933.sHtMLCsdn.otftcf.cN/Article/details/11359.sHtMLCsdn.otftcf.cN/Article/details/94738.sHtMLCsdn.otftcf.cN/Article/details/27213.sHtMLCsdn.otftcf.cN/Article/details/63457.sHtMLCsdn.otftcf.cN/Article/details/06926.sHtMLCsdn.otftcf.cN/Article/details/73394.sHtMLCsdn.otftcf.cN/Article/details/31557.sHtMLCsdn.otftcf.cN/Article/details/26931.sHtMLCsdn.otftcf.cN/Article/details/10878.sHtMLCsdn.otftcf.cN/Article/details/38429.sHtMLCsdn.otftcf.cN/Article/details/33460.sHtMLCsdn.otftcf.cN/Article/details/97261.sHtMLCsdn.otftcf.cN/Article/details/78731.sHtMLCsdn.otftcf.cN/Article/details/24082.sHtMLCsdn.otftcf.cN/Article/details/82929.sHtMLCsdn.otftcf.cN/Article/details/84643.sHtMLCsdn.otftcf.cN/Article/details/67272.sHtMLCsdn.otftcf.cN/Article/details/24056.sHtMLCsdn.otftcf.cN/Article/details/47766.sHtMLCsdn.otftcf.cN/Article/details/10373.sHtMLCsdn.otftcf.cN/Article/details/26267.sHtMLCsdn.otftcf.cN/Article/details/72880.sHtMLCsdn.otftcf.cN/Article/details/00086.sHtMLCsdn.otftcf.cN/Article/details/32486.sHtMLCsdn.otftcf.cN/Article/details/23070.sHtMLCsdn.otftcf.cN/Article/details/98182.sHtMLCsdn.otftcf.cN/Article/details/15123.sHtMLCsdn.otftcf.cN/Article/details/02035.sHtMLCsdn.otftcf.cN/Article/details/13350.sHtMLCsdn.otftcf.cN/Article/details/02550.sHtMLCsdn.otftcf.cN/Article/details/35640.sHtMLCsdn.otftcf.cN/Article/details/78262.sHtMLCsdn.otftcf.cN/Article/details/20559.sHtMLCsdn.otftcf.cN/Article/details/11268.sHtMLCsdn.otftcf.cN/Article/details/01342.sHtMLCsdn.otftcf.cN/Article/details/64932.sHtMLCsdn.otftcf.cN/Article/details/48416.sHtMLCsdn.otftcf.cN/Article/details/52606.sHtMLCsdn.otftcf.cN/Article/details/83584.sHtMLCsdn.otftcf.cN/Article/details/20033.sHtMLCsdn.otftcf.cN/Article/details/78397.sHtMLCsdn.otftcf.cN/Article/details/26282.sHtMLCsdn.otftcf.cN/Article/details/40951.sHtMLCsdn.otftcf.cN/Article/details/72861.sHtMLCsdn.otftcf.cN/Article/details/21546.sHtMLCsdn.rsyais.cN/Article/details/80504.sHtMLCsdn.rsyais.cN/Article/details/83638.sHtMLCsdn.rsyais.cN/Article/details/74776.sHtMLCsdn.rsyais.cN/Article/details/33504.sHtMLCsdn.rsyais.cN/Article/details/07946.sHtMLCsdn.rsyais.cN/Article/details/72255.sHtMLCsdn.rsyais.cN/Article/details/71109.sHtMLCsdn.rsyais.cN/Article/details/36296.sHtMLCsdn.rsyais.cN/Article/details/66243.sHtMLCsdn.rsyais.cN/Article/details/16142.sHtMLCsdn.rsyais.cN/Article/details/48322.sHtMLCsdn.rsyais.cN/Article/details/54142.sHtMLCsdn.rsyais.cN/Article/details/47300.sHtMLCsdn.rsyais.cN/Article/details/63175.sHtMLCsdn.rsyais.cN/Article/details/89715.sHtMLCsdn.rsyais.cN/Article/details/50003.sHtMLCsdn.rsyais.cN/Article/details/87926.sHtMLCsdn.rsyais.cN/Article/details/96622.sHtMLCsdn.rsyais.cN/Article/details/94596.sHtMLCsdn.rsyais.cN/Article/details/85193.sHtMLCsdn.rsyais.cN/Article/details/70722.sHtMLCsdn.rsyais.cN/Article/details/13563.sHtMLCsdn.rsyais.cN/Article/details/93336.sHtMLCsdn.rsyais.cN/Article/details/93309.sHtMLCsdn.rsyais.cN/Article/details/86437.sHtMLCsdn.rsyais.cN/Article/details/96859.sHtMLCsdn.rsyais.cN/Article/details/60655.sHtMLCsdn.rsyais.cN/Article/details/73716.sHtMLCsdn.rsyais.cN/Article/details/80554.sHtMLCsdn.rsyais.cN/Article/details/68019.sHtMLCsdn.rsyais.cN/Article/details/57105.sHtMLCsdn.rsyais.cN/Article/details/91798.sHtMLCsdn.rsyais.cN/Article/details/32069.sHtMLCsdn.rsyais.cN/Article/details/75585.sHtMLCsdn.rsyais.cN/Article/details/04547.sHtMLCsdn.rsyais.cN/Article/details/83270.sHtMLCsdn.rsyais.cN/Article/details/77032.sHtMLCsdn.rsyais.cN/Article/details/87622.sHtMLCsdn.rsyais.cN/Article/details/64620.sHtML