秒杀项目真的是早有耳闻,可以说是大火有一阵子,因为这其中涉及高并发、数据库、缓存,更有甚者还有分布式、分库分表、集群等。
这次有机会跟着视频学习了一点秒杀系统,这里做个总结
参考
https://www.bilibili.com/video/BV1CE411s7xN
https://www.bilibili.com/video/BV13a4y1t7Wh
基础环境和工具
IDEA 2020 + JDK8
SpringBoot 2.x
虚拟机CentOS上的 MySQL 5.7x + Redis 6.x
Mybatis
Lombok
Navicat (MySQL可视化工具)
Redis Desktop Manager (Redis可视化工具)
MobaXterm (SSH工具)
JMeter (压力测试工具)
Postman/Chrome
pom.xml 基础依赖
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 <dependencies > <dependency > <groupId > org.springframework.boot</groupId > <artifactId > spring-boot-starter-data-redis</artifactId > </dependency > <dependency > <groupId > mysql</groupId > <artifactId > mysql-connector-java</artifactId > </dependency > <dependency > <groupId > org.mybatis.spring.boot</groupId > <artifactId > mybatis-spring-boot-starter</artifactId > <version > 2.1.4</version > </dependency > <dependency > <groupId > com.alibaba</groupId > <artifactId > druid-spring-boot-starter</artifactId > <version > 1.2.4</version > </dependency > <dependency > <groupId > org.springframework.boot</groupId > <artifactId > spring-boot-starter-web</artifactId > </dependency > <dependency > <groupId > org.projectlombok</groupId > <artifactId > lombok</artifactId > <optional > true</optional > </dependency > <dependency > <groupId > org.springframework.boot</groupId > <artifactId > spring-boot-starter-test</artifactId > <scope > test</scope > </dependency > </dependencies >
application
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 server: port: 8090 spring: datasource: url: jdbc:mysql://192.168.1.106:3306/seckill driver-class-name: com.mysql.cj.jdbc.Driver username: root password: root redis: host: 192.168 .1 .106 port: 6379 password: root mybatis: mapper-locations: classpath:mapper/*.xml type-aliases-package: com.wnh.entity configuration: map-underscore-to-camel-case: true
1 2 3 logging.level.root =info logging.level.com.wnh.dao =debug
项目框架
项目结构
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 DROP TABLE IF EXISTS `stock`;CREATE TABLE `stock` ( `sid` int (11 ) NOT NULL AUTO_INCREMENT COMMENT '商品id' , `name` varchar (20 ) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT '商品名称' , `total` int (11 ) NOT NULL COMMENT '库存' , `sale` int (11 ) NOT NULL COMMENT '已售' , `version` int (11 ) NOT NULL COMMENT '版本号' , PRIMARY KEY (`sid`) USING BTREE ) ENGINE = InnoDB AUTO_INCREMENT = 4 CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic ; DROP TABLE IF EXISTS `stock_order`;CREATE TABLE `stock_order` ( `oid` int (11 ) NOT NULL AUTO_INCREMENT, `sid` int (11 ) NOT NULL , `create_time` timestamp (0 ) NOT NULL DEFAULT CURRENT_TIMESTAMP (0 ) ON UPDATE CURRENT_TIMESTAMP (0 ), PRIMARY KEY (`oid`) USING BTREE ) ENGINE = InnoDB AUTO_INCREMENT = 4749 CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic ;
注意这里订单不能命名为order,MySQL保留字错误
Entity
Stock
1 2 3 4 5 6 7 8 9 10 11 12 13 @Data @AllArgsConstructor @NoArgsConstructor @ToString @Accessors(chain = true) public class Stock { private Integer sid; private String name; private Integer total; private Integer sale; private Integer version; }
Order
1 2 3 4 5 6 7 8 9 10 @Data @AllArgsConstructor @NoArgsConstructor @ToString @Accessors(chain = true) public class Order { private Integer oid; private Integer sid; private Date createTime; }
Mapper
商品Mapper
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 <!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" > <mapper namespace ="com.wnh.dao.StockMapper" > <update id ="updateSale" parameterType ="stock" > update stock set sale=sale + 1 where sid = #{sid} and total > sale </update > <update id ="updateSaleWithVersion" parameterType ="stock" > update stock set sale=sale + 1, version=version + 1 where sid = #{sid} and version = #{version} </update > <select id ="checkStock" parameterType ="int" resultType ="stock" > select sid, name, total, sale, version from stock where sid = #{id} </select > <select id ="listStocks" resultType ="stock" > select sid, total, sale from stock </select > </mapper >
可以发现这里有两个不同的更新操作,两个都是利用数据库的乐观锁实现的简单的并发处理
订单Mapper
1 2 3 4 5 6 7 8 9 10 11 <!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" > <mapper namespace ="com.wnh.dao.OrderMapper" > <insert id ="createOrder" parameterType ="order" useGeneratedKeys ="true" keyProperty ="oid" > insert into stock_order values (#{oid}, #{sid}, #{createTime}) </insert > </mapper >
Service
StockService
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 @Service @Transactional public class StockServiceImpl implements StockService { @Autowired private StockMapper stockMapper; @Override public List<Stock> listStocks () { return stockMapper.listStocks(); } @Override public Stock checkStock (Integer id) { Stock stock = stockMapper.checkStock(id); if (stock.getSale().equals(stock.getTotal())) { throw new RuntimeException ("库存不足" ); } return stock; } @Override public int updateSale (Stock stock) { return stockMapper.updateSale(stock); } }
OrderService
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 @Service @Transactional public class OrderServiceImpl implements OrderService { @Autowired private StockService stockService; @Autowired private OrderMapper orderMapper; @Override public int kill (Integer id) { Stock stock = stockService.checkStock(id); int up = stockService.updateSale(stock); if (up == 0 ) { throw new RuntimeException ("库存不足" ); } return createOrder(stock); } private Integer createOrder (Stock stock) { Order order = new Order (); order.setSid(stock.getSid()).setCreateTime(new Date ()); orderMapper.createOrder(order); return order.getOid(); } }
关键接口是 OrderService 的 kill 方法
Controller
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 @RestController @RequestMapping("stock") public class BuyController { @Autowired private StockService stockService; @Autowired private OrderService orderService; @GetMapping("kill") public String kill (Integer id) { System.out.println("秒杀商品id = " + id); try { int orderId = orderService.kill(id); return "秒杀成功,订单id为:" + orderId; } catch (Exception e) { e.printStackTrace(); return e.getMessage(); } } }
启动项目测试
初始数据库数据
商品表
1
iphone8
100
0
0
2
p40
5
0
0
3
k30
200
0
0
订单表为空
postman一次请求
一次请求后
sale+1,变为1,增加了一条订单,其他没有变化
JMeter测试
JMeter测试
sale变为100,订单总数100条,Throughput为100-200/sec
总体没出现大问题,但是Throughput不尽人意
恢复原数据把 JMeter 参数稍稍一改,1000不变,Loop Cout
变为10,也就是总共 10000
次请求,这次整个测试过程变得很慢,最终Throughput也停留在
30-40/sec,当然这需要优化
加入Redis
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 @RestController @RequestMapping("stock") public class BuyController { @Autowired private StockService stockService; @Autowired private StringRedisTemplate stringRedisTemplate; @Autowired private OrderService orderService; @PostConstruct public void init () { List<Stock> stocks = stockService.listStocks(); for (Stock stock : stocks) { stringRedisTemplate.opsForValue().set(Constants.REDIS_STOCK_LAST + stock.getSid(), stock.getTotal() - stock.getSale() + "" ); } } @GetMapping("kill") public String kill (Integer id) { Long increment = stringRedisTemplate.opsForValue().decrement(Constants.REDIS_STOCK_LAST + id); if (increment < 0 ) { stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + id); return "商品已售完" ; } System.out.println("秒杀商品id = " + id); try { int orderId = orderService.kill(id); return "秒杀成功,订单id为:" + orderId; } catch (Exception e) { stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + id); e.printStackTrace(); return e.getMessage(); } } }
启动系统
查到 Redis 里已经存入数据,Postman测试没有问题
JMeter 测试过程很快,但 Throughput 提升一点
二级缓存
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 @RestController @RequestMapping("stock") public class BuyController { @Autowired private StockService stockService; @Autowired private StringRedisTemplate stringRedisTemplate; @Autowired private OrderService orderService; private static ConcurrentHashMap<Integer, Boolean> stockSoldOutMap = new ConcurrentHashMap <>(); @PostConstruct public void init () { List<Stock> stocks = stockService.listStocks(); for (Stock stock : stocks) { stringRedisTemplate.opsForValue().set(Constants.REDIS_STOCK_LAST + stock.getSid(), stock.getTotal() - stock.getSale() + "" ); } } @GetMapping("kill") public String kill (Integer id) { if (stockSoldOutMap.get(id) != null ) { return "商品已售完" ; } Long increment = stringRedisTemplate.opsForValue().decrement(Constants.REDIS_STOCK_LAST + id); if (increment < 0 ) { stockSoldOutMap.put(id, true ); stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + id); return "商品已售完" ; } System.out.println("秒杀商品id = " + id); try { int orderId = orderService.kill(id); return "秒杀成功,订单id为:" + orderId; } catch (Exception e) { stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + id); if (stockSoldOutMap.get(id) != null ) { stockSoldOutMap.remove(id); } e.printStackTrace(); return e.getMessage(); } } }
再次测试,因为通过JVM内存缓存了是否售空,所以系统还能再提升一些。
令牌桶限流
依赖
1 2 3 4 5 6 <dependency > <groupId > com.google.guava</groupId > <artifactId > guava</artifactId > <version > 30.1-jre</version > </dependency >
Controller
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 private RateLimiter rateLimiter = RateLimiter.create(40 );@GetMapping("sale") public String sale (Integer id) { if (!rateLimiter.tryAcquire(2 , TimeUnit.SECONDS)) { System.out.println("当前请求被限流,直接抛弃,无法调用后续秒杀逻辑..." ); } System.out.println("处理业务............." ); return "抢购成功" ; }
完整Controller
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 @RestController @RequestMapping("stock") @Slf4j public class BuyController { @Autowired private StockService stockService; @Autowired private StringRedisTemplate stringRedisTemplate; @Autowired private OrderService orderService; private static ConcurrentHashMap<Integer, Boolean> stockSoldOutMap = new ConcurrentHashMap <>(); private RateLimiter rateLimiter = RateLimiter.create(20 ); @GetMapping("sale") public String sale (Integer id) { if (!rateLimiter.tryAcquire(2 , TimeUnit.SECONDS)) { System.out.println("当前请求被限流,直接抛弃,无法调用后续秒杀逻辑..." ); return "抢购失败" ; } System.out.println("处理业务............." ); return "抢购成功" ; } @GetMapping("killtoken") public String killtoken (Integer id) { if (!rateLimiter.tryAcquire(2 , TimeUnit.SECONDS)) { System.out.println("当前请求被限流,直接抛弃,无法调用后续秒杀逻辑..." ); return "抢购失败" ; } return kill(id); } @PostConstruct public void init () { List<Stock> stocks = stockService.listStocks(); for (Stock stock : stocks) { stringRedisTemplate.opsForValue().set(Constants.REDIS_STOCK_LAST + stock.getSid(), stock.getTotal() - stock.getSale() + "" ); } } @GetMapping("kill") public String kill (Integer id) { if (stockSoldOutMap.get(id) != null ) { return "商品已售完" ; } Long increment = stringRedisTemplate.opsForValue().decrement(Constants.REDIS_STOCK_LAST + id); if (increment < 0 ) { stockSoldOutMap.put(id, true ); stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + id); return "商品已售完" ; } System.out.println("秒杀商品id = " + id); try { int orderId = orderService.kill(id); return "秒杀成功,订单id为:" + orderId; } catch (Exception e) { stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + id); if (stockSoldOutMap.get(id) != null ) { stockSoldOutMap.remove(id); } e.printStackTrace(); return e.getMessage(); } } }
测试 killtoken
接口,可以发现在一定情况下是卖不完的,虽然请求数大于库存数,这时仍然没有超卖问题
问题
规定时间段内可抢购,其他时间不能
恶意抓包获取接口,脚本抢购
单个用户限制抢购
限时抢购
利用 Redis 过期时间,设置抢购时间
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 @GetMapping("kill") public String kill (Integer id) { if (!stringRedisTemplate.hasKey(Constants.REDIS_STOCK_KILL + id)) { System.out.println("秒杀活动已结束..." ); return "秒杀活动已结束" ; } if (stockSoldOutMap.get(id) != null ) { return "商品已售完" ; } Long increment = stringRedisTemplate.opsForValue().decrement(Constants.REDIS_STOCK_LAST + id); if (increment < 0 ) { stockSoldOutMap.put(id, true ); stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + id); return "商品已售完" ; } System.out.println("秒杀商品id = " + id); try { int orderId = orderService.kill(id); return "秒杀成功,订单id为:" + orderId; } catch (Exception e) { stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + id); if (stockSoldOutMap.get(id) != null ) { stockSoldOutMap.remove(id); } e.printStackTrace(); return e.getMessage(); } }
测试
设置1号商品5秒内可抢购
set stock_kill_1 1 EX 5
JMeter测试正常,部分因令牌桶限流,部分因超过抢购时间直接返回
防脚本验证
用户表
1 2 3 4 5 6 CREATE TABLE `user ` ( `uid` int (11 ) NOT NULL AUTO_INCREMENT COMMENT '用户id' , `name` varchar (20 ) NOT NULL COMMENT '用户名' , `password` varchar (20 ) NOT NULL COMMENT '密码' , PRIMARY KEY (`uid`) ) ENGINE= InnoDB AUTO_INCREMENT= 2 DEFAULT CHARSET= utf8
加入
User
1 2 3 4 5 6 7 @Data @ToString public class User { private Integer uid; private String name; private String password; }
UserMapper
1 2 3 4 @Mapper public interface UserMapper { User findUserById (Integer id) ; }
UserMapper.xml
1 2 3 4 5 6 7 8 9 10 11 12 <!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" > <mapper namespace ="com.wnh.dao.UserMapper" > <select id ="findUserById" parameterType ="int" resultType ="user" > select uid, name, password from user where uid = #{id} </select > </mapper >
Controller新加入
1 2 3 4 5 6 7 8 9 10 11 12 @RequestMapping("md5") public String getMD5 (Integer sid, Integer uid) { String md5; try { md5 = orderService.getMD5(sid, uid); } catch (Exception e) { e.printStackTrace(); return "获取md5失败:" + e.getMessage(); } return "获取的MD5为:" + md5; }
OrderService
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 @Autowired private UserMapper userMapper;@Autowired StringRedisTemplate stringRedisTemplate; @Override public String getMD5 (Integer sid, Integer uid) { User user = userMapper.findUserById(uid); if (user == null ) { throw new RuntimeException ("用户不存在!" ); } log.info("用户信息:[{}]" , user.toString()); Stock stock = stockService.checkStock(sid); if (stock == null ) { throw new RuntimeException ("商品不存在!" ); } log.info("商品信息:[{}]" , stock.toString()); String hashKey = "KEY_" + uid + "_" + sid; String key = DigestUtils.md5DigestAsHex((uid + "!Qr*#3" + sid).getBytes()); stringRedisTemplate.opsForValue().set(hashKey, key, 120 , TimeUnit.SECONDS); log.info("Redis写入:[{}]-[{}]" , hashKey, key); return key; }
postman测试接口为http://localhost:8090/stock/md5?sid=1&uid=1
结果为:获取的MD5为:6f2c1a31e4b297c4f85c166c09009ec6
请求MD5串
控制台
控制台日志
查看Redis,没有问题
查看Redis
看到设置了120秒的过期时间
改造加入验证
Controller
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 @GetMapping("killtokenmd5") public String killtokenmd5 (Integer sid, Integer uid, String md5) { if (!rateLimiter.tryAcquire(2 , TimeUnit.SECONDS)) { System.out.println("当前请求被限流,直接抛弃,无法调用后续秒杀逻辑..." ); return "抢购失败" ; } if (stockSoldOutMap.get(sid) != null ) { return "商品已售完" ; } Long increment = stringRedisTemplate.opsForValue().decrement(Constants.REDIS_STOCK_LAST + sid); if (increment < 0 ) { stockSoldOutMap.put(sid, true ); stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + sid); return "商品已售完" ; } System.out.println("秒杀商品id = " + sid); try { int orderId = orderService.killtokenmd5(sid, uid, md5); return "秒杀成功,订单id为:" + orderId; } catch (Exception e) { stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + sid); if (stockSoldOutMap.get(sid) != null ) { stockSoldOutMap.remove(sid); } e.printStackTrace(); return e.getMessage(); } }
Service
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 @Override public int killtokenmd5 (Integer sid, Integer uid, String md5) { String hashKey = "KEY_" + uid + "_" + sid; if (!md5.equals(stringRedisTemplate.opsForValue().get(hashKey))) { throw new RuntimeException ("当前请求不合法,请稍后再试!" ); } Stock stock = stockService.checkStock(sid); int up = stockService.updateSale(stock); if (up == 0 ) { throw new RuntimeException ("库存不足" ); } return createOrder(stock); }
测试,先请求获取 md5 接口使 Redis 存在该
md5,在过期时间内请求新的秒杀接口,不同于前,需要加上用户 id 和 md5
用以验证
正确结果如下,若不先请求 md5,则无法利用 Redis 验证导致失败,错误的
md5 同样 失败
带MD5正确请求
用户限制
限制用户访问频率
利用 Redis 超时时间和 incr 操作限制访问频率
UserService
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 @Service @Transactional @Slf4j public class UserServiceImpl implements UserService { @Autowired StringRedisTemplate stringRedisTemplate; @Override public long saveUserView (Integer uid) { String limitKey = "LIMIT" + "_" + uid; String limitNum = stringRedisTemplate.opsForValue().get(limitKey); long limit = -1 ; if (limitNum == null ) { stringRedisTemplate.opsForValue().set(limitKey, "0" , 3600 , TimeUnit.SECONDS); } else { limit = stringRedisTemplate.opsForValue().increment(limitKey); } return limit; } @Override public boolean getUserView (Integer uid) { String limitKey = "LIMIT" + "_" + uid; String limitNum = stringRedisTemplate.opsForValue().get(limitKey); if (limitNum == null ) { log.error("该用户没用申请验证值记录,疑似异常" ); } return Integer.parseInt(limitNum) <= 10 ; } }
Controller
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 @GetMapping("killtokenmd5limit") public String killtokenmd5limit (Integer sid, Integer uid, String md5) { if (!rateLimiter.tryAcquire(2 , TimeUnit.SECONDS)) { System.out.println("当前请求被限流,直接抛弃,无法调用后续秒杀逻辑..." ); return "抢购失败" ; } if (stockSoldOutMap.get(sid) != null ) { return "商品已售完" ; } Long increment = stringRedisTemplate.opsForValue().decrement(Constants.REDIS_STOCK_LAST + sid); if (increment < 0 ) { stockSoldOutMap.put(sid, true ); stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + sid); return "商品已售完" ; } System.out.println("秒杀商品id = " + sid); try { long view = userService.saveUserView(uid); log.info("用户已访问次数:[{}]" , view); boolean isAllowed= userService.getUserView(uid); if (!isAllowed) { log.info("购买失败,超过访问频率!" ); return "购买失败,超过访问频率!" ; } int orderId = orderService.killtokenmd5(sid, uid, md5); return "秒杀成功,订单id为:" + orderId; } catch (Exception e) { stringRedisTemplate.opsForValue().increment(Constants.REDIS_STOCK_LAST + sid); if (stockSoldOutMap.get(sid) != null ) { stockSoldOutMap.remove(sid); } e.printStackTrace(); return e.getMessage(); } }
测试,依然先获取md5,在用 JMeter
测试,接口及参数/stock/killtokenmd5limit?sid=1&uid=1&md5=6f2c1a31e4b297c4f85c166c09009ec6
访问
20
次结果如下,因为限制10次,所以10次后被限制,也就是限制为10/(h*u)单用户每小时限制访问10次
控制台日志
总结
这仅仅是一个小demo,还有好多要学习
继续努力吧