Merge pull request #84 from Lost-MSth/dev

Update to v2.10.2

.gitignore

@@ -1,5 +1,20 @@
+# log files
 *.log
+*.log.*
+
+# SSL cert
+*.pem
+*.key
+
+# sqlite3 database
+*.db
+*.db.bak*
+
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]
-*$py.class
+*$py.class
+
+# setting/config files
+latest version/config/
+latest version/config.py

README.md

@@ -1,7 +1,9 @@
 # Arcaea-server
+
 一个微型的Arcaea本地服务器  A small local server for Arcaea
 
 ## 简介 Introduction
+
 这是基于Python以及Flask的微型本地Arcaea服务器，可以模拟游戏的主要功能。这可能是我第一次写这种大程序，若有不妥之处，敬请谅解。  
 
 本程序主要用于学习研究，不得用于任何商业行为，否则后果自负，这不是强制要求，只是一个提醒与警告。  
@@ -14,7 +16,9 @@ This procedure is mainly used for study and research, and shall not be used for
 > It looks stupid, but it works!
 
 ## 特性 Features
+
 有以下 We have：
+
 - 登录、注册 Login and registration
 - 多设备登录 Multi device login
 - 成绩上传 Score upload
@@ -24,7 +28,7 @@ This procedure is mainly used for study and research, and shall not be used for
 - 段位系统 Course system
 - Link Play
 - 好友系统 Friends
-- 数据同步 Data synchronization
+- 云端存档 Cloud save
 - 爬梯 Climbing steps
 - 自定义世界模式 Customizable World Mode
 - 自定义歌曲下载 Customizable songs download
@@ -40,66 +44,52 @@ This procedure is mainly used for study and research, and shall not be used for
 - 服务器日志 Server log
 
 没有以下 We don't have：
+
 - 服务器安全性保证 Server security assurance
 
 可能有问题 There may be problems：
+
 - Recent 30
 - 一些歌曲的解锁 Some songs' unlocking
 - 同设备多共存登录 Multiple app logins on the same device
 
 ## 说明 Statement
+
 只是很有趣，用处探索中。  
 It is just so interesting. What it can do is under exploration.
 
-
 ## 下载 Download
+
 [这里 Here](https://github.com/Lost-MSth/Arcaea-server/releases)
 
 [Arcaea-CN official](https://arcaea.lowiro.com/zh)  
-[Arcaea-lowi.ro](https://lowi.ro)  
-[Arcaea-RhyDown](https://rhydown.com)
+[Arcaea-Konmai Academy](https://616.sb)  
 
 ## 更新日志 Update log
+
 只保留最新版本 Only keep the latest version.
 
 > 提醒：更新时请注意保留原先的数据库，以防数据丢失。
 >
-> Tips: When updating, please keep the original database in case of data loss. 
+> Tips: When updating, please keep the original database in case of data loss.
 
+### Version 2.10.2
 
-### Version 2.10.0
-- 适用于Arcaea 4.1.0版本 For Arcaea 4.1.0
-- 新搭档 **咲姬** 已解锁 Unlock the character **Saki**.
-- 新搭档 **刹那** 已解锁 Unlock the character **Setsuna**.
-- 完善了日志系统 Improve the log system.
-- 现在可以利用`songlist`确保`3.aff`以外文件不被下载 Now you can use `songlist` to ensure that files other than `3.aff` should not be downloaded. [#60](https://github.com/Lost-MSth/Arcaea-server/issues/60)
-- 适配v4.0.0以下版本的客户端云存档 Ensure that the clients under v4.0.0 can upload the cloud save.
-- 优化数据库索引 Optimize database indices.
-- 尝试确保LinkPlay服务器的线程安全，现在此功能将作为独立服务端 Try to ensure thread safety in LinkPlay server. Now this function will be served as an independent server.
-- 对API接口的分数列表添加歌曲名 Add song names for getting the score list in API.
-- 为下载错误增添HTTP状态码 Add HTTP status code when meeting download error.
-
-- 修复iOS客户端因世界模式地图数据闪退的问题 Fix a bug when world maps' data don't have some unnecessary parts the client of iOS may break down.
-- 修复API接口无数据`GET`请求导致报错的问题 Fix a bug that `GET` requests without data will report an error in API. [#50](https://github.com/Lost-MSth/Arcaea-server/issues/50)
-- 修复`aggregate`请求无法捕获内部错误的问题 Fix a bug that `aggregate` requests will get an error when the inner function raises an error.
-- 修复因错误设置主键导致课程模式谱面无法相同的问题 Fix a bug that the charts of a course cannot be the same because of the incorrect primary keys.
-- 修复无谱面数据时世界排名分计算出错的问题 Fix a bug that global ranking scores cannot be calculated if there are no chart in the database. [#61](https://github.com/Lost-MSth/Arcaea-server/issues/61)
-- 修复条件满足但隐藏段位依然无法解锁的问题 Fix a bug that the hidden courses cannot appear even if their requirements are satisfied.
-- 修复Finale挑战中某些无法解锁的问题 Fix a bug that something of the finale challenge cannot be unlocked.
-- 修复用户物品数量无法为0的问题，此问题导致了一些购买操作异常 Fix a bug that the users' items will not be zero, which will disturb some purchase operations.
-- 修复角色等级能超过最大等级的问题 Fix a bug that the level of the character can exceed the max level.
-- 修复使用`以太之滴`升级角色时应答不正确的问题 Fix a bug that the response is incorrect when upgrading the characters by `generic core`.
-- 修复`源韵强化`数值显示不正确的问题 Fix a bug that the `prog boost` shows the incorrect value.
-- 修复世界模式奖励可能被重复发放的问题 Fix a bug that the rewards can be get repeatedly in World Mode.
-- 修复世界Boss的第二管血量无法削减的问题 Fix a bug that second tube of blood of the world boss won't change.
-- 修复我的排名显示不正确的问题 Fix a bug that `my rank` doesn't work correctly.
-- 修复在歌曲结束后无法及时轮换房主的问题 Fix a bug that the room host will be changed late when finishing a song.
-
+- 适用于Arcaea 4.1.7版本 For Arcaea 4.1.7
+- 新搭档 **红（冬日）** 已解锁 Unlock the character **Kou(Winter)**.
+- 新增记录数据库来记录全部的游玩历史分数 Add a log database to record all playing scores.
+- 新增设置选项，可选择阻止或接受unranked成绩 Add a config option that can be used to forbid unranked scores.
+- 为自定义异常添加简明的warning日志 Add brief warning logs for custom exceptions.
+- 修复flask应用启动前出现异常，日志无法正确地指出异常的问题 Fix a bug that if an exception is raised before flask app runs, logger will not work well.
+- 现在初始化文件中JSON文件可以是模块支持的其它编码格式 Now initial files can be other encoding types which are supported by JSON module.
+- `run.bat`在报错时会停下而不是一闪而过了 Make the `run.bat` script pause when meeting an error. #82
+- 新增API接口查询单谱排行 Add an API endpoint for getting the rank list of a song's chart. #81
 
 ## 运行环境与依赖 Running environment and requirements
+
 - Windows/Linux/Mac OS/Android...
-- Python 3
-- Flask module >= 2.0, Cryptography module >= 35.0.0
+- Python >= 3.6
+- Flask module >= 2.0, Cryptography module >= 3.0.0, limits >= 2.7.0
 - Charles, IDA, proxy app... (optional)
 
 <!--
@@ -109,34 +99,41 @@ It is just so interesting. What it can do is under exploration.
 -->
 
 ## 使用说明 Instruction for use
+
 [中文](https://github.com/Lost-MSth/Arcaea-server/wiki/%E4%BD%BF%E7%94%A8%E8%AF%B4%E6%98%8E)  
 [English](https://github.com/Lost-MSth/Arcaea-server/wiki/Instruction-for-use)
 
 ## 注意 Attentions
+
 [中文](https://github.com/Lost-MSth/Arcaea-server/wiki/%E6%B3%A8%E6%84%8F)  
 [English](https://github.com/Lost-MSth/Arcaea-server/wiki/Attentions)
 
-
 ## Q&A
+
 [中文/English](https://github.com/Lost-MSth/Arcaea-server/wiki/Q&A)
 
-
 ## 鸣谢 Thanks
-~~歌曲数据库来自 Using song database from
-[BotArcAPI releases](https://github.com/TheSnowfield/BotArcAPI/releases)~~
+
+歌曲数据库来自 Using song database from
+~~[BotArcAPI releases](https://github.com/TheSnowfield/BotArcAPI/releases)~~
+[ArcaeaSongDatabase](https://github.com/Arcaea-Infinity/ArcaeaSongDatabase)
+
 > 从v2.9开始不再提供歌曲数据  
 > Since v2.9, song data will not be provided.
 
 网站图标来自 Using favicon from [black fate - てんてん - pixiv](https://www.pixiv.net/artworks/82374369)
 
 ## 联系方式 Contact
+
 如有必要，可以联系本人 Contact me if necessary  
 邮箱 Email：th84292@foxmail.com
 
 ## 支持一下 Support me
+
 生活不易。 Life is not easy.  
 [支付宝 Alipay](https://github.com/Lost-MSth/Arcaea-server/blob/master/pic/Alipay.jpg)  
 [微信 WeChat](https://github.com/Lost-MSth/Arcaea-server/blob/master/pic/WeChat.png)
 
 ## 使用许可 Use license
+
 [MIT](LICENSE) © Lost

latest version/api/api_auth.py

@@ -1,11 +1,13 @@
 from functools import wraps
 from traceback import format_exc
+from base64 import b64decode
+from json import loads
 
 from core.api_user import APIUser
+from core.config_manager import Config
 from core.error import ArcError, NoAccess, PostError
 from core.sql import Connect
 from flask import current_app
-from setting import Config
 
 from .api_code import error_return
 
@@ -25,20 +27,17 @@ def role_required(request, powers=[]):
                 return error_return(PostError('No token', api_error_code=-1), 401)
 
             user = APIUser()
-            if Config.API_TOKEN == request.headers['Token'] and Config.API_TOKEN != '':
-                user.user_id = 0
-            elif powers == []:
-                # 无powers则非本地权限（API_TOKEN规定的）无法访问
-                return error_return(NoAccess('No permission', api_error_code=-1), 403)
-            else:
-                with Connect() as c:
+            with Connect() as c:
+                user.c = c
+                if Config.API_TOKEN == request.headers['Token'] and Config.API_TOKEN != '':
+                    user.set_role_system()
+                else:
                     try:
-                        user.c = c
                         user.select_user_id_from_api_token(
                             request.headers['Token'])
                         user.select_role_and_powers()
 
-                        if not any([y in [x.power_name for x in user.role.powers] for y in powers]):
+                        if not any(user.role.has_power(y) for y in powers):
                             return error_return(NoAccess('No permission', api_error_code=-1), 403)
                     except ArcError as e:
                         return error_return(e, 401)
@@ -63,17 +62,24 @@ def request_json_handle(request, required_keys=[], optional_keys=[]):
         def wrapped_view(*args, **kwargs):
 
             data = {}
-            if not request.data:
-                return view(data, *args, **kwargs)
+            if request.data:
+                json_data = request.json
+            else:
+                if request.method == 'GET' and 'query' in request.args:
+                    # 处理axios没法GET传data的问题
+                    json_data = loads(
+                        b64decode(request.args['query']).decode())
+                else:
+                    json_data = {}
 
             for key in required_keys:
-                if key not in request.json:
-                    return error_return(PostError('Missing parameter: ' + key, api_error_code=-100))
-                data[key] = request.json[key]
+                if key not in json_data:
+                    return error_return(PostError(f'Missing parameter: {key}', api_error_code=-100))
+                data[key] = json_data[key]
 
             for key in optional_keys:
-                if key in request.json:
-                    data[key] = request.json[key]
+                if key in json_data:
+                    data[key] = json_data[key]
 
             return view(data, *args, **kwargs)
 

latest version/api/api_code.py

@@ -6,20 +6,23 @@ default_error = ArcError('Unknown Error')
 
 CODE_MSG = {
     0: '',
-    -1: 'See status code',
+    -1: 'See status code',  # 基础错误
     -2: 'No data',
-    -3: 'No data or user',
-    -4: 'No user_id',
-    -100: 'Wrong post data',
-    -101: 'Wrong data type',
-    -102: 'Wrong query parameter',
-    -103: 'Wrong sort parameter',
-    -104: 'Wrong sort order parameter',
-    -200: 'No permission',
+    -3: 'No data or user',  # 不确定是无数据还是无用户
+    -100: 'Invalid post data',  # 1xx数据错误
+    -101: 'Invalid data type',
+    -102: 'Invalid query parameter',
+    -103: 'Invalid sort parameter',
+    -104: 'Invalid sort order parameter',
+    -105: 'Invalid URL parameter',
+    -110: 'Invalid user_id',
+    -200: 'No permission',  # 2xx用户相关错误
     -201: 'Wrong username or password',
     -202: 'User is banned',
-    -203: 'Username exists',
-    -204: 'Email address exists',
+    -203: 'Too many login attempts',
+    -210: 'Username exists',
+    -211: 'Email address exists',
+    -212: 'User code exists',
     -999: 'Unknown error'
 }
 

latest version/api/songs.py

@@ -1,4 +1,5 @@
-from core.error import NoData
+from core.error import NoData, InputError
+from core.rank import RankList
 from core.song import Song
 from core.sql import Connect, Query, Sql
 from flask import Blueprint, request
@@ -30,7 +31,7 @@ def songs_get(data, user):
     B = ['song_id', 'name', 'rating_pst',
          'rating_prs', 'rating_ftr', 'rating_byn']
     with Connect() as c:
-        query = Query(A, A, B).from_data(data)
+        query = Query(A, A, B).from_dict(data)
         x = Sql(c).select('chart', query=query)
         r = []
         for i in x:
@@ -40,3 +41,27 @@ def songs_get(data, user):
             raise NoData(api_error_code=-2)
 
         return success_return([x.to_dict() for x in r])
+
+
+@bp.route('/<string:song_id>/<int:difficulty>/rank', methods=['GET'])
+@role_required(request, ['select', 'select_song_rank', 'select_song_rank_top'])
+@request_json_handle(request, optional_keys=['limit'])
+@api_try
+def songs_song_difficulty_rank_get(data, user, song_id, difficulty):
+    '''查询歌曲某个难度的成绩排行榜，和游戏内接口相似，只允许limit'''
+    if difficulty not in [0, 1, 2, 3]:
+        raise InputError('Difficulty must be 0, 1, 2 or 3')
+    limit = data.get('limit', 20)
+    if not isinstance(limit, int):
+        raise InputError('Limit must be int')
+    if user.role.only_has_powers(['select_song_rank_top'], ['select', 'select_song_rank']):
+        # 限制低权限只能查询前20名
+        if limit > 20 or limit < 0:
+            limit = 20
+    with Connect() as c:
+        rank_list = RankList(c)
+        rank_list.song.set_chart(song_id, difficulty)
+        rank_list.limit = limit
+        # 不检查歌曲是否存在，不存在返回的是空列表
+        rank_list.select_top()
+        return success_return(rank_list.to_dict_list())

latest version/api/users.py

@@ -1,7 +1,8 @@
 from core.error import InputError, NoAccess, NoData
 from core.score import Potential, UserScoreList
 from core.sql import Connect, Query, Sql
-from core.user import UserInfo, UserRegister
+from core.user import UserChanger, UserInfo, UserRegister
+from core.api_user import APIUser
 from flask import Blueprint, request
 
 from .api_auth import api_try, request_json_handle, role_required
@@ -15,7 +16,7 @@ bp = Blueprint('users', __name__, url_prefix='/users')
 @role_required(request, ['change'])
 @request_json_handle(request, ['name', 'password', 'email'])
 @api_try
-def users_post(data, _):
+def users_post(data, user):
     '''注册一个用户'''
     with Connect() as c:
         new_user = UserRegister(c)
@@ -36,7 +37,7 @@ def users_get(data, user):
     B = ['user_id', 'name', 'user_code', 'join_date',
          'rating_ptt', 'time_played', 'ticket', 'world_rank_score']
     with Connect() as c:
-        query = Query(A, A, B).from_data(data)
+        query = Query(A, A, B).from_dict(data)
         x = Sql(c).select('user', query=query)
         r = []
         for i in x:
@@ -50,7 +51,7 @@ def users_get(data, user):
             'name': x.name,
             'join_date': x.join_date,
             'user_code': x.user_code,
-            'rating_ptt': x.rating_ptt/100,
+            'rating_ptt': x.rating_ptt,
             'character_id': x.character.character_id,
             'is_char_uncapped': x.character.is_uncapped,
             'is_char_uncapped_override': x.character.is_uncapped_override,
@@ -65,9 +66,9 @@ def users_get(data, user):
 def users_user_get(user, user_id):
     '''查询用户信息'''
     if user_id <= 0:
-        return error_return(InputError(api_error_code=-4))
+        return error_return(InputError(api_error_code=-110))
     # 查别人需要select权限
-    if user_id != user.user_id and user.user_id != 0 and not user.role.has_power('select'):
+    if user_id != user.user_id and not user.role.has_power('select'):
         return error_return(NoAccess('No permission', api_error_code=-1), 403)
 
     with Connect() as c:
@@ -75,21 +76,56 @@ def users_user_get(user, user_id):
         return success_return(u.to_dict())
 
 
+@bp.route('/<int:user_id>', methods=['PUT'])
+@role_required(request, ['change'])
+@request_json_handle(request, optional_keys=['name', 'password', 'user_code', 'ticket', 'email'])
+@api_try
+def users_user_put(data, user, user_id):
+    '''修改一个用户'''
+    with Connect() as c:
+        u = UserChanger(c, user_id)
+        r = {}
+        r['user_id'] = user_id
+        if 'name' in data:
+            u.set_name(data['name'])
+            r['name'] = u.name
+        if 'password' in data:
+            u.set_password(data['password'])
+            r['password'] = u.hash_pwd
+        if 'email' in data:
+            u.set_email(data['email'])
+            r['email'] = u.email
+        if 'user_code' in data:
+            u.set_user_code(data['user_code'])
+            r['user_code'] = u.user_code
+        if 'ticket' in data:
+            if not isinstance(data['ticket'], int):
+                raise InputError('Ticket must be int')
+            u.ticket = data['ticket']
+            r['ticket'] = u.ticket
+        if r:
+            u.update_columns(d=r)
+        return success_return(r)
+
+
 @bp.route('/<int:user_id>/b30', methods=['GET'])
 @role_required(request, ['select', 'select_me'])
 @api_try
 def users_user_b30_get(user, user_id):
     '''查询用户b30'''
     if user_id <= 0:
-        return error_return(InputError(api_error_code=-4))
+        return error_return(InputError(api_error_code=-110))
     # 查别人需要select权限
-    if user_id != user.user_id and user.user_id != 0 and not user.role.has_power('select'):
+    if user_id != user.user_id and not user.role.has_power('select'):
         return error_return(NoAccess('No permission', api_error_code=-1), 403)
 
     with Connect() as c:
         x = UserScoreList(c, UserInfo(c, user_id))
         x.query.limit = 30
         x.select_from_user()
+        if not x.scores:
+            raise NoData(
+                f'No best30 data of user `{user_id}`', api_error_code=-3)
         x.select_song_name()
         r = x.to_dict_list()
         rating_sum = sum([i.rating for i in x.scores])
@@ -103,15 +139,18 @@ def users_user_b30_get(user, user_id):
 def users_user_best_get(data, user, user_id):
     '''查询用户所有best成绩'''
     if user_id <= 0:
-        return error_return(InputError(api_error_code=-4))
+        return error_return(InputError(api_error_code=-110))
     # 查别人需要select权限
-    if user_id != user.user_id and user.user_id != 0 and not user.role.has_power('select'):
+    if user_id != user.user_id and not user.role.has_power('select'):
         return error_return(NoAccess('No permission', api_error_code=-1), 403)
 
     with Connect() as c:
         x = UserScoreList(c, UserInfo(c, user_id))
-        x.query.from_data(data)
+        x.query.from_dict(data)
         x.select_from_user()
+        if not x.scores:
+            raise NoData(
+                f'No best score data of user `{user_id}`', api_error_code=-3)
         r = x.to_dict_list()
         return success_return({'user_id': user_id, 'data': r})
 
@@ -123,11 +162,32 @@ def users_user_r30_get(user, user_id):
     '''查询用户r30'''
 
     if user_id <= 0:
-        return error_return(InputError(api_error_code=-4))
+        return error_return(InputError(api_error_code=-110))
     # 查别人需要select权限
-    if user_id != user.user_id and user.user_id != 0 and not user.role.has_power('select'):
+    if user_id != user.user_id and not user.role.has_power('select'):
         return error_return(NoAccess('No permission', api_error_code=-1), 403)
 
     with Connect() as c:
         p = Potential(c, UserInfo(c, user_id))
         return success_return({'user_id': user_id, 'r10_ptt': p.recent_10 / 10, 'data': p.recent_30_to_dict_list()})
+
+
+@bp.route('/<int:user_id>/role', methods=['GET'])
+@role_required(request, ['select', 'select_me'])
+@api_try
+def users_user_role_get(user, user_id):
+    '''查询用户role和powers'''
+
+    if user_id <= 0:
+        return error_return(InputError(api_error_code=-110))
+
+    if user_id == user.user_id:
+        return success_return({'user_id': user.user_id, 'role': user.role.role_id, 'powers': [i.power_id for i in user.role.powers]})
+    # 查别人需要select权限
+    if not user.role.has_power('select'):
+        return error_return(NoAccess('No permission', api_error_code=-1), 403)
+
+    with Connect() as c:
+        x = APIUser(c, user_id)
+        x.select_role_and_powers()
+        return success_return({'user_id': x.user_id, 'role': x.role.role_id, 'powers': [i.power_id for i in x.role.powers]})

latest version/config.example.py

@@ -1,6 +1,7 @@
 class Config():
     '''
-    This is the setting file. You can change some parameters here.
+    This is the example setting file.
+    The user's setting file's name is `config.py`.
     '''
 
     '''
@@ -9,7 +10,7 @@ class Config():
     Host and port of your server
     '''
     HOST = '0.0.0.0'
-    PORT = '80'
+    PORT = 80
     '''
     --------------------
     '''

latest version/core/api_user.py

@@ -2,57 +2,62 @@ from hashlib import sha256
 from os import urandom
 from time import time
 
-from .error import NoAccess, NoData, UserBan
+from .config_manager import Config
+from .error import NoAccess, NoData, RateLimit, UserBan
+from .limiter import ArcLimiter
 from .user import UserOnline
 
 
 class Power:
     def __init__(self, c=None):
         self.c = c
-        self.power_id: int = None
-        self.power_name: str = None
+        self.power_id: str = None
         self.caption: str = None
 
     @classmethod
     def from_dict(cls, d: dict, c=None) -> 'Power':
         p = cls(c)
         p.power_id = d['power_id']
-        p.power_name = d['power_name']
         p.caption = d['caption']
         return p
 
-    def select_from_name(self, power_name: str) -> 'Power':
-        pass
-
 
 class Role:
     def __init__(self, c=None):
         self.c = c
-        self.role_id: int = None
-        self.role_name: str = None
+        self.role_id: str = None
         self.caption: str = None
 
         self.powers: list = None
 
-    def has_power(self, power_name: str) -> bool:
+    def has_power(self, power_id: str) -> bool:
         '''判断role是否有power'''
+        return any(power_id == i.power_id for i in self.powers)
+
+    def only_has_powers(self, power_ids: list, anti_power_ids: list = None) -> bool:
+        '''判断role是否全有power_ids里的power，且没有anti_power_ids里的任何一个power'''
+        flags = [False] * len(power_ids)
+        if anti_power_ids is None:
+            anti_power_ids = []
         for i in self.powers:
-            if i.power_name == power_name:
-                return True
-        return False
+            if i.power_id in anti_power_ids:
+                return False
+            for j, k in enumerate(power_ids):
+                if i.power_id == k:
+                    flags[j] = True
+        return all(flags)
 
     def select_from_id(self, role_id: int = None) -> 'Role':
         '''用role_id查询role'''
         if role_id is not None:
             self.role_id = role_id
-        self.c.execute('''select role_name, caption from role where role_id = :a''',
+        self.c.execute('''select caption from role where role_id = :a''',
                        {'a': self.role_id})
         x = self.c.fetchone()
         if x is None:
             raise NoData('The role `%s` does not exist.' %
                          self.role_id, api_error_code=-200)
-        self.role_name = x[0]
-        self.caption = x[1]
+        self.caption = x[0]
         return self
 
     def select_powers(self) -> None:
@@ -63,10 +68,12 @@ class Role:
         x = self.c.fetchall()
         for i in x:
             self.powers.append(Power.from_dict(
-                {'power_id': i[0], 'power_name': i[1], 'caption': i[2]}, self.c))
+                {'power_id': i[0], 'caption': i[1]}, self.c))
 
 
 class APIUser(UserOnline):
+    limiter = ArcLimiter(Config.API_LOGIN_RATE_LIMIT, 'api_login')
+
     def __init__(self, c=None, user_id=None) -> None:
         super().__init__(c, user_id)
         self.api_token: str = None
@@ -74,6 +81,13 @@ class APIUser(UserOnline):
 
         self.ip: str = None
 
+    def set_role_system(self) -> None:
+        '''设置为最高权限用户，API接口'''
+        self.user_id = 0
+        self.role = Role(self.c)
+        self.role.role_id = 'system'
+        self.role.select_powers()
+
     def select_role(self) -> None:
         '''查询user的role'''
         self.c.execute('''select role_id from user_role where user_id = :a''',
@@ -82,10 +96,9 @@ class APIUser(UserOnline):
         self.role = Role(self.c)
         if x is None:
             # 默认role为user
-            self.role.role_id = 1
+            self.role.role_id = 'user'
         else:
-            self.role.role_id = int(x[0])
-        self.role.select_from_id()
+            self.role.role_id = x[0]
 
     def select_role_and_powers(self) -> None:
         '''查询user的role，以及role的powers'''
@@ -113,6 +126,9 @@ class APIUser(UserOnline):
             self.password = password
         if ip is not None:
             self.ip = ip
+        if not self.limiter.hit(name):
+            raise RateLimit('Too many login attempts', api_error_code=-205)
+
         self.c.execute('''select user_id, password from user where name = :a''', {
                        'a': self.name})
         x = self.c.fetchone()

latest version/core/character.py

@@ -1,6 +1,6 @@
-from setting import Config
-from .error import ArcError, InputError, NoData, ItemNotEnough
+from .config_manager import Config
 from .constant import Constant
+from .error import ArcError, InputError, ItemNotEnough, NoData
 from .item import Item, ItemCore
 
 

latest version/core/config_manager.py

@@ -0,0 +1,96 @@
+class Config:
+    '''
+    Default config
+    '''
+
+    HOST = '0.0.0.0'
+    PORT = 80
+
+    DEPLOY_MODE = 'flask_multithread'
+    USE_PROXY_FIX = False
+    USE_CORS = False
+
+    GAME_API_PREFIX = '/join/21'
+
+    ALLOW_APPVERSION = []  # list[str]
+
+    SET_LINKPLAY_SERVER_AS_SUB_PROCESS = True
+
+    LINKPLAY_HOST = '0.0.0.0'
+    LINKPLAY_UDP_PORT = 10900
+    LINKPLAY_TCP_PORT = 10901
+    LINKPLAY_AUTHENTICATION = 'my_link_play_server'
+    LINKPLAY_DISPLAY_HOST = ''
+
+    SSL_CERT = ''
+    SSL_KEY = ''
+
+    IS_APRILFOOLS = True
+
+    WORLD_RANK_MAX = 200
+
+    AVAILABLE_MAP = []  # list[str]
+
+    USERNAME = 'admin'
+    PASSWORD = 'admin'
+
+    SECRET_KEY = '1145141919810'
+
+    API_TOKEN = ''
+
+    DOWNLOAD_LINK_PREFIX = ''
+
+    DOWNLOAD_USE_NGINX_X_ACCEL_REDIRECT = False
+    NGINX_X_ACCEL_REDIRECT_PREFIX = '/nginx_download/'
+
+    DOWNLOAD_TIMES_LIMIT = 3000
+    DOWNLOAD_TIME_GAP_LIMIT = 1000
+
+    LOGIN_DEVICE_NUMBER_LIMIT = 1
+    ALLOW_LOGIN_SAME_DEVICE = False
+    ALLOW_BAN_MULTIDEVICE_USER_AUTO = True
+
+    ALLOW_SCORE_WITH_NO_SONG = True
+
+    ALLOW_INFO_LOG = False
+    ALLOW_WARNING_LOG = False
+
+    DEFAULT_MEMORIES = 0
+
+    UPDATE_WITH_NEW_CHARACTER_DATA = True
+
+    CHARACTER_FULL_UNLOCK = True
+    WORLD_SONG_FULL_UNLOCK = True
+    WORLD_SCENERY_FULL_UNLOCK = True
+
+    SAVE_FULL_UNLOCK = False
+
+    # ------------------------------------------
+
+    # You can change this to make another PTT mechanism.
+    BEST30_WEIGHT = 1 / 40
+    RECENT10_WEIGHT = 1 / 40
+
+    MAX_FRIEND_COUNT = 50
+
+    WORLD_MAP_FOLDER_PATH = './database/map/'
+    SONG_FILE_FOLDER_PATH = './database/songs/'
+    SONGLIST_FILE_PATH = './database/songs/songlist'
+    SQLITE_DATABASE_PATH = './database/arcaea_database.db'
+    SQLITE_DATABASE_BACKUP_FOLDER_PATH = './database/backup/'
+    DATABASE_INIT_PATH = './database/init/'
+    SQLITE_LOG_DATABASE_PATH = './database/arcaea_log.db'
+
+    GAME_LOGIN_RATE_LIMIT = '30/5 minutes'
+    API_LOGIN_RATE_LIMIT = '10/5 minutes'
+
+
+class ConfigManager:
+
+    @staticmethod
+    def load(config) -> None:
+        for k, v in config.__dict__.items():
+            if k.startswith('__') or k.endswith('__'):
+                continue
+            if hasattr(Config, k):
+                setattr(Config, k, v)

latest version/core/constant.py

@@ -1,4 +1,6 @@
-from setting import Config
+from .config_manager import Config
+
+ARCAEA_SERVER_VERSION = 'v2.10.2'
 
 
 class Constant:
@@ -9,6 +11,8 @@ class Constant:
 
     STAMINA_RECOVER_TICK = 1800000
 
+    COURSE_STAMINA_COST = 4
+
     CORE_EXP = 250
 
     LEVEL_STEPS = {1: 0, 2: 50, 3: 100, 4: 150, 5: 200, 6: 300, 7: 450, 8: 650, 9: 900, 10: 1200, 11: 1600, 12: 2100, 13: 2700, 14: 3400, 15: 4200, 16: 5100,
@@ -19,19 +23,19 @@ class Constant:
     AYU_UNCAP_BONUS_PROGRESS = 5
     SKILL_FATALIS_WORLD_LOCKED_TIME = 3600000
 
-    MAX_FRIEND_COUNT = 50
+    MAX_FRIEND_COUNT = Config.MAX_FRIEND_COUNT
 
     MY_RANK_MAX_LOCAL_POSITION = 5
     MY_RANK_MAX_GLOBAL_POSITION = 9999
 
-    # You can change this to make another PTT mechanism.
-    BEST30_WEIGHT = 1 / 40
-    RECENT10_WEIGHT = 1 / 40
+    BEST30_WEIGHT = Config.BEST30_WEIGHT
+    RECENT10_WEIGHT = Config.RECENT10_WEIGHT
 
-    WORLD_MAP_FOLDER_PATH = './database/map/'
-    SONG_FILE_FOLDER_PATH = './database/songs/'
-    SONGLIST_FILE_PATH = './database/songs/songlist'
-    SQLITE_DATABASE_PATH = './database/arcaea_database.db'
+    WORLD_MAP_FOLDER_PATH = Config.WORLD_MAP_FOLDER_PATH
+    SONG_FILE_FOLDER_PATH = Config.SONG_FILE_FOLDER_PATH
+    SONGLIST_FILE_PATH = Config.SONGLIST_FILE_PATH
+    SQLITE_DATABASE_PATH = Config.SQLITE_DATABASE_PATH
+    SQLITE_LOG_DATABASE_PATH = Config.SQLITE_LOG_DATABASE_PATH
 
     DOWNLOAD_TIMES_LIMIT = Config.DOWNLOAD_TIMES_LIMIT
     DOWNLOAD_TIME_GAP_LIMIT = Config.DOWNLOAD_TIME_GAP_LIMIT
@@ -45,8 +49,6 @@ class Constant:
     LINKPLAY_UDP_PORT = Config.LINKPLAY_UDP_PORT
     LINKPLAY_AUTHENTICATION = Config.LINKPLAY_AUTHENTICATION
 
-    COURSE_STAMINA_COST = 4
-
     # Well, I can't say a word when I see this.
     FINALE_SWITCH = [
         (0x0015F0, 0x00B032), (0x014C9A, 0x014408), (0x062585, 0x02783B),
@@ -93,3 +95,8 @@ class Constant:
         (0X5D6FC5, 0xab97ef), (0X237206D, 0xdfef2), (0XA3DEE,
                                                      0x6CB300), (0XA35687B, 0xE456CDEA)
     ]
+
+    DATABASE_MIGRATE_TABLES = ['user', 'friend', 'best_score', 'recent30', 'user_world', 'item', 'user_item', 'purchase', 'purchase_item', 'user_save',
+                               'login', 'present', 'user_present', 'present_item', 'redeem', 'user_redeem', 'redeem_item', 'api_login', 'chart', 'user_course', 'user_char', 'user_role']
+
+    UPDATE_WITH_NEW_CHARACTER_DATA = Config.UPDATE_WITH_NEW_CHARACTER_DATA

latest version/core/download.py

@@ -7,6 +7,7 @@ from flask import url_for
 
 from .constant import Constant
 from .error import NoAccess
+from .limiter import ArcLimiter
 from .user import User
 from .util import get_file_md5, md5
 
@@ -19,29 +20,69 @@ def get_song_file_md5(song_id: str, file_name: str) -> str:
     return get_file_md5(path)
 
 
-def initialize_songfile():
-    '''初始化歌曲数据的md5信息'''
-    get_song_file_md5.cache_clear()
-    x = DownloadList()
-    x.url_flag = False
-    x.add_songs()
-    del x
+class SonglistParser:
+    '''songlist文件解析器'''
 
+    FILE_NAMES = ['0.aff', '1.aff', '2.aff', '3.aff',
+                  'base.ogg', '3.ogg', 'video.mp4', 'video_audio.ogg']
+    songs: dict = {}  # {song_id: value, ...}
+    # value: bit 76543210
+    # 7: video_audio.ogg
+    # 6: video.mp4
+    # 5: 3.ogg
+    # 4: base.ogg
+    # 3: 3.aff
+    # 2: 2.aff
+    # 1: 1.aff
+    # 0: 0.aff
 
-@lru_cache()
-def get_only_3_song_ids():
-    '''初始化只能下载byd相关的歌曲id'''
-    if not os.path.isfile(Constant.SONGLIST_FILE_PATH):
-        return []
-    only_3_song_ids = []
-    data = []
-    with open(Constant.SONGLIST_FILE_PATH, 'r', encoding='utf-8') as f:
-        data = loads(f.read())['songs']
-    for x in data:
-        if 'remote_dl' not in x or 'remote_dl' in x and not x['remote_dl']:
-            if any(i['ratingClass'] == 3 for i in x['difficulties']):
-                only_3_song_ids.append(x['id'])
-    return only_3_song_ids
+    def __init__(self, path=Constant.SONGLIST_FILE_PATH) -> None:
+        self.path = path
+        self.data: list = []
+        self.parse()
+
+    @staticmethod
+    def is_available_file(song_id: str, file_name: str) -> bool:
+        '''判断文件是否允许被下载'''
+        if song_id not in SonglistParser.songs:
+            # songlist没有，则只限制文件名
+            return file_name in SonglistParser.FILE_NAMES
+        rule = SonglistParser.songs[song_id]
+        for i in range(8):
+            if file_name == SonglistParser.FILE_NAMES[i] and rule & (1 << i) != 0:
+                return True
+        return False
+
+    def parse_one(self, song: dict) -> dict:
+        '''解析单个歌曲'''
+        if not 'id' in song:
+            return None
+        r = 0
+        if 'remote_dl' in song and song['remote_dl']:
+            r |= 16
+            for i in song.get('difficulties', []):
+                if i['ratingClass'] == 3 and i.get('audioOverride', False):
+                    r |= 32
+                r |= 1 << i['ratingClass']
+        else:
+            if any(i['ratingClass'] == 3 for i in song.get('difficulties', [])):
+                r |= 8
+
+        if 'additional_files' in song:
+            if 'video.mp4' in song['additional_files']:
+                r |= 64
+            if 'video_audio.ogg' in song['additional_files']:
+                r |= 128
+        return {song['id']: r}
+
+    def parse(self) -> None:
+        '''解析songlist文件'''
+        if not os.path.isfile(self.path):
+            return
+        with open(self.path, 'r', encoding='utf-8') as f:
+            self.data = loads(f.read()).get('songs', [])
+        for x in self.data:
+            self.songs.update(self.parse_one(x))
 
 
 class UserDownload:
@@ -50,8 +91,11 @@ class UserDownload:
         properties: `user` - `User`类或子类的实例
     '''
 
-    def __init__(self, c=None, user=None) -> None:
-        self.c = c
+    limiter = ArcLimiter(
+        str(Constant.DOWNLOAD_TIMES_LIMIT) + '/day', 'download')
+
+    def __init__(self, c_m=None, user=None) -> None:
+        self.c_m = c_m
         self.user = user
 
         self.song_id: str = None
@@ -60,19 +104,13 @@ class UserDownload:
         self.token: str = None
         self.token_time: int = None
 
-    def clear_user_download(self) -> None:
-        self.c.execute(
-            '''delete from user_download where user_id = :a and time <= :b''', {'a': self.user.user_id, 'b': int(time()) - 24*3600})
-
     @property
     def is_limited(self) -> bool:
         '''是否达到用户最大下载量'''
         if self.user is None:
             self.select_for_check()
-        self.c.execute(
-            '''select count(*) from user_download where user_id = :a''', {'a': self.user.user_id})
-        y = self.c.fetchone()
-        return y is not None and y[0] > Constant.DOWNLOAD_TIMES_LIMIT
+
+        return not self.limiter.test(str(self.user.user_id))
 
     @property
     def is_valid(self) -> bool:
@@ -81,19 +119,19 @@ class UserDownload:
             self.select_for_check()
         return int(time()) - self.token_time <= Constant.DOWNLOAD_TIME_GAP_LIMIT
 
-    def insert_user_download(self) -> None:
-        '''记录下载信息'''
-        self.c.execute('''insert into user_download values(:a,:b,:c)''', {
-                       'a': self.user.user_id, 'c': self.token, 'b': int(time())})
+    def download_hit(self) -> bool:
+        '''下载次数+1，返回成功与否bool值'''
+        return self.limiter.hit(str(self.user.user_id))
 
     def select_for_check(self) -> None:
         '''利用token、song_id、file_name查询其它信息'''
-        self.c.execute('''select user_id, time from download_token where song_id=? and file_name=? and token = ? limit 1;''',
-                       (self.song_id, self.file_name, self.token))
+        self.c_m.execute('''select user_id, time from download_token where song_id=? and file_name=? and token = ? limit 1;''',
+                         (self.song_id, self.file_name, self.token))
 
-        x = self.c.fetchone()
+        x = self.c_m.fetchone()
         if not x:
-            raise NoAccess('The token `%s` is not valid.' % self.token, status=403)
+            raise NoAccess('The token `%s` is not valid.' %
+                           self.token, status=403)
         self.user = User()
         self.user.user_id = x[0]
         self.token_time = x[1]
@@ -101,11 +139,11 @@ class UserDownload:
     def generate_token(self) -> None:
         self.token_time = int(time())
         self.token = md5(str(self.user.user_id) + self.song_id +
-                         self.file_name + str(self.token_time))
+                         self.file_name + str(self.token_time) + str(os.urandom(8)))
 
     def insert_download_token(self) -> None:
         '''将数据插入数据库，让这个下载链接可用'''
-        self.c.execute('''insert into download_token values(:a,:b,:c,:d,:e)''', {
+        self.c_m.execute('''insert or replace into download_token values(:a,:b,:c,:d,:e)''', {
             'a': self.user.user_id, 'b': self.song_id, 'c': self.file_name, 'd': self.token, 'e': self.token_time})
 
     @property
@@ -113,14 +151,14 @@ class UserDownload:
         '''生成下载链接'''
         if self.token is None:
             self.generate_token()
-            self.insert_download_token()
+            # self.insert_download_token() # 循环插入速度慢，改为executemany
         if Constant.DOWNLOAD_LINK_PREFIX:
             prefix = Constant.DOWNLOAD_LINK_PREFIX
             if prefix[-1] != '/':
                 prefix += '/'
-            return prefix + self.song_id + '/' + self.file_name + '?t=' + self.token
+            return f'{prefix}{self.song_id}/{self.file_name}?t={self.token}'
         else:
-            return url_for('download', file_path=self.song_id + '/' + self.file_name, t=self.token, _external=True)
+            return url_for('download', file_path=f'{self.song_id}/{self.file_name}', t=self.token, _external=True)
 
     @property
     def hash(self) -> str:
@@ -133,8 +171,8 @@ class DownloadList(UserDownload):
         properties: `user` - `User`类或子类的实例
     '''
 
-    def __init__(self, c=None, user=None) -> None:
-        super().__init__(c, user)
+    def __init__(self, c_m=None, user=None) -> None:
+        super().__init__(c_m, user)
 
         self.song_ids: list = None
         self.url_flag: bool = None
@@ -142,68 +180,107 @@ class DownloadList(UserDownload):
         self.downloads: list = []
         self.urls: dict = {}
 
-    def clear_download_token_from_song(self, song_id: str) -> None:
-        self.c.execute('''delete from download_token where user_id=:a and song_id=:b''', {
-            'a': self.user.user_id, 'b': song_id})
+    @classmethod
+    def initialize_cache(cls) -> None:
+        '''初始化歌曲数据缓存，包括md5、文件目录遍历、解析songlist'''
+        SonglistParser()
+        x = cls()
+        x.url_flag = False
+        x.add_songs()
+        del x
+
+    @staticmethod
+    def clear_all_cache() -> None:
+        '''清除所有歌曲文件有关缓存'''
+        get_song_file_md5.cache_clear()
+        DownloadList.get_one_song_file_names.cache_clear()
+        DownloadList.get_all_song_ids.cache_clear()
+        SonglistParser.songs = {}
+
+    def clear_download_token(self) -> None:
+        '''清除过期下载链接'''
+        self.c_m.execute('''delete from download_token where time<?''',
+                         (int(time()) - Constant.DOWNLOAD_TIME_GAP_LIMIT,))
+
+    def insert_download_tokens(self) -> None:
+        '''插入所有下载链接'''
+        self.c_m.executemany('''insert or replace into download_token values(?,?,?,?,?)''', [(
+            self.user.user_id, x.song_id, x.file_name, x.token, x.token_time) for x in self.downloads])
+
+    @staticmethod
+    @lru_cache(maxsize=2048)
+    def get_one_song_file_names(song_id: str) -> list:
+        '''获取一个歌曲文件夹下的所有合法文件名，有lru缓存'''
+        r = []
+        for i in os.listdir(os.path.join(Constant.SONG_FILE_FOLDER_PATH, song_id)):
+            if os.path.isfile(os.path.join(Constant.SONG_FILE_FOLDER_PATH, song_id, i)) and SonglistParser.is_available_file(song_id, i):
+                r.append(i)
+        return r
 
     def add_one_song(self, song_id: str) -> None:
-        if self.url_flag:
-            self.clear_download_token_from_song(song_id)
-        dir_list = os.listdir(os.path.join(
-            Constant.SONG_FILE_FOLDER_PATH, song_id))
 
         re = {}
-        for i in dir_list:
-            if os.path.isfile(os.path.join(Constant.SONG_FILE_FOLDER_PATH, song_id, i)) and i in ['0.aff', '1.aff', '2.aff', '3.aff', 'base.ogg', '3.ogg', 'video.mp4', 'video_audio.ogg']:
-                if song_id in get_only_3_song_ids() and i not in ['3.aff', '3.ogg']:
-                    continue
-                x = UserDownload(self.c, self.user)
-                # self.downloads.append(x) # 这实际上没有用
-                x.song_id = song_id
-                x.file_name = i
-                if i == 'base.ogg':
-                    if 'audio' not in re:
-                        re['audio'] = {}
+        for i in self.get_one_song_file_names(song_id):
+            x = UserDownload(self.c_m, self.user)
+            self.downloads.append(x)
+            x.song_id = song_id
+            x.file_name = i
+            if i == 'base.ogg':
+                if 'audio' not in re:
+                    re['audio'] = {}
 
-                    re['audio']["checksum"] = x.hash
-                    if self.url_flag:
-                        re['audio']["url"] = x.url
-                elif i == '3.ogg':
-                    if 'audio' not in re:
-                        re['audio'] = {}
+                re['audio']["checksum"] = x.hash
+                if self.url_flag:
+                    re['audio']["url"] = x.url
+            elif i == '3.ogg':
+                if 'audio' not in re:
+                    re['audio'] = {}
 
-                    if self.url_flag:
-                        re['audio']['3'] = {"checksum": x.hash, "url": x.url}
-                    else:
-                        re['audio']['3'] = {"checksum": x.hash}
-                elif i == 'video.mp4' or i == 'video_audio.ogg':
-                    if 'additional_files' not in re:
-                        re['additional_files'] = []
-
-                    if self.url_flag:
-                        re['additional_files'].append(
-                            {"checksum": x.hash, "url": x.url, 'file_name': i})
-                    else:
-                        re['additional_files'].append(
-                            {"checksum": x.hash, 'file_name': i})
+                if self.url_flag:
+                    re['audio']['3'] = {"checksum": x.hash, "url": x.url}
                 else:
-                    if 'chart' not in re:
-                        re['chart'] = {}
+                    re['audio']['3'] = {"checksum": x.hash}
+            elif i == 'video.mp4' or i == 'video_audio.ogg':
+                if 'additional_files' not in re:
+                    re['additional_files'] = []
 
-                    if self.url_flag:
-                        re['chart'][i[0]] = {"checksum": x.hash, "url": x.url}
-                    else:
-                        re['chart'][i[0]] = {"checksum": x.hash}
+                if self.url_flag:
+                    re['additional_files'].append(
+                        {"checksum": x.hash, "url": x.url, 'file_name': i})
+                else:
+                    re['additional_files'].append(
+                        {"checksum": x.hash, 'file_name': i})
+            else:
+                if 'chart' not in re:
+                    re['chart'] = {}
+
+                if self.url_flag:
+                    re['chart'][i[0]] = {"checksum": x.hash, "url": x.url}
+                else:
+                    re['chart'][i[0]] = {"checksum": x.hash}
 
         self.urls.update({song_id: re})
 
+    @staticmethod
+    @lru_cache()
+    def get_all_song_ids() -> list:
+        '''获取全歌曲文件夹列表，有lru缓存'''
+        return list(filter(lambda x: os.path.isdir(os.path.join(Constant.SONG_FILE_FOLDER_PATH, x)), os.listdir(Constant.SONG_FILE_FOLDER_PATH)))
+
     def add_songs(self, song_ids: list = None) -> None:
         '''添加一个或多个歌曲到下载列表，若`song_ids`为空，则添加所有歌曲'''
         if song_ids is not None:
             self.song_ids = song_ids
 
-        x = self.song_ids if self.song_ids else os.listdir(
-            Constant.SONG_FILE_FOLDER_PATH)
-        for i in x:
-            if os.path.isdir(os.path.join(Constant.SONG_FILE_FOLDER_PATH, i)):
+        if not self.song_ids:
+            self.song_ids = self.get_all_song_ids()
+            for i in self.song_ids:
                 self.add_one_song(i)
+        else:
+            for i in self.song_ids:
+                if os.path.isdir(os.path.join(Constant.SONG_FILE_FOLDER_PATH, i)):
+                    self.add_one_song(i)
+
+        if self.url_flag:
+            self.clear_download_token()
+            self.insert_download_tokens()

latest version/core/error.py

@@ -1,91 +1,111 @@
 class ArcError(Exception):
     def __init__(self, message=None, error_code=108, api_error_code=-999, extra_data=None, status=200) -> None:
-        self.message = message
-        self.error_code = error_code
-        self.api_error_code = api_error_code
+        self.message: str = message
+        self.error_code: int = error_code
+        self.api_error_code: int = api_error_code
         self.extra_data = extra_data
-        self.status = status
+        self.status: int = status
 
     def __str__(self) -> str:
         return repr(self.message)
 
 
 class InputError(ArcError):
-    # 输入类型错误
+    '''输入类型错误'''
+
     def __init__(self, message=None, error_code=108, api_error_code=-100, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class DataExist(ArcError):
-    # 数据存在
+    '''数据存在'''
     pass
 
 
 class NoData(ArcError):
-    # 数据不存在
+    '''数据不存在'''
+
     def __init__(self, message=None, error_code=108, api_error_code=-2, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class PostError(ArcError):
-    # 缺少输入
+    '''缺少输入'''
+
     def __init__(self, message=None, error_code=108, api_error_code=-100, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class UserBan(ArcError):
-    # 用户封禁
+    '''用户封禁'''
+
     def __init__(self, message=None, error_code=121, api_error_code=-202, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class ItemNotEnough(ArcError):
-    # 物品数量不足
+    '''物品数量不足'''
+
     def __init__(self, message=None, error_code=-6, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class ItemUnavailable(ArcError):
-    # 物品不可用
+    '''物品不可用'''
+
     def __init__(self, message=None, error_code=-6, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class RedeemUnavailable(ArcError):
-    # 兑换码不可用
+    '''兑换码不可用'''
+
     def __init__(self, message=None, error_code=505, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class MapLocked(ArcError):
-    # 地图锁定
+    '''地图锁定'''
+
     def __init__(self, message=None, error_code=108, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class StaminaNotEnough(ArcError):
-    # 体力不足
+    '''体力不足'''
+
     def __init__(self, message=None, error_code=107, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class TicketNotEnough(ArcError):
-    # 记忆源点不足
+    '''记忆源点不足'''
+
     def __init__(self, message=None, error_code=-6, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class FriendError(ArcError):
-    # 好友系统出错
+    '''好友系统出错'''
+
     def __init__(self, message=None, error_code=108, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class NoAccess(ArcError):
-    # 无权限
-    pass
+    '''无权限'''
+
+    def __init__(self, message=None, error_code=108, api_error_code=-999, extra_data=None, status=403) -> None:
+        super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class Timeout(ArcError):
-    # 超时
+    '''超时'''
     pass
+
+
+class RateLimit(ArcError):
+    '''频率达到限制'''
+
+    def __init__(self, message=None, error_code=108, api_error_code=-999, extra_data=None, status=429) -> None:
+        super().__init__(message, error_code, api_error_code, extra_data, status)

latest version/core/init.py

@@ -0,0 +1,299 @@
+import os
+import sys
+from importlib import import_module
+from json import load
+from shutil import copy, copy2
+from time import time
+from traceback import format_exc
+
+from core.config_manager import Config
+from core.constant import ARCAEA_SERVER_VERSION
+from core.course import Course
+from core.download import DownloadList
+from core.purchase import Purchase
+from core.sql import Connect, DatabaseMigrator, MemoryDatabase
+from core.user import UserRegister
+from core.util import try_rename
+
+
+class DatabaseInit:
+    def __init__(self, db_path: str = Config.SQLITE_DATABASE_PATH, init_folder_path: str = Config.DATABASE_INIT_PATH) -> None:
+        self.db_path = db_path
+        self.init_folder_path = init_folder_path
+        self.c = None
+        self.init_data = None
+
+    @property
+    def sql_path(self) -> str:
+        return os.path.join(self.init_folder_path, 'tables.sql')
+
+    @property
+    def course_path(self) -> str:
+        return os.path.join(self.init_folder_path, 'courses.json')
+
+    @property
+    def pack_path(self) -> str:
+        return os.path.join(self.init_folder_path, 'packs.json')
+
+    @property
+    def single_path(self) -> str:
+        return os.path.join(self.init_folder_path, 'singles.json')
+
+    def table_init(self) -> None:
+        '''初始化数据库结构'''
+        with open(self.sql_path, 'r') as f:
+            self.c.executescript(f.read())
+        self.c.execute('''insert into config values("version", :a);''', {
+            'a': ARCAEA_SERVER_VERSION})
+
+    def character_init(self) -> None:
+        '''初始化搭档信息'''
+        for i in range(0, len(self.init_data.char)):
+            skill_requires_uncap = 1 if i == 2 else 0
+            if i in [0, 1, 2, 4, 13, 26, 27, 28, 29, 36, 21, 42, 43, 11, 12, 19, 5]:
+                max_level = 30
+                uncapable = 1
+            else:
+                max_level = 20
+                uncapable = 0
+            sql = '''insert into character values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)'''
+            self.c.execute(sql, (i, self.init_data.char[i], max_level, self.init_data.frag1[i], self.init_data.prog1[i], self.init_data.overdrive1[i], self.init_data.frag20[i], self.init_data.prog20[i], self.init_data.overdrive20[i],
+                                 self.init_data.frag30[i], self.init_data.prog30[i], self.init_data.overdrive30[i], self.init_data.skill_id[i], self.init_data.skill_unlock_level[i], skill_requires_uncap, self.init_data.skill_id_uncap[i], self.init_data.char_type[i], uncapable))
+
+        self.c.execute('''insert into character values(?,?,20,?,?,?,?,?,?,?,?,?,?,?,?,?,?,0)''', (
+            99, 'shirahime', 38, 33, 28, 66, 58, 50, 66, 58, 50, 'frags_preferred_song', 0, 0, '', 0))
+
+        for i in self.init_data.char_core:
+            self.c.executemany('''insert into char_item values(?,?,'core',?)''', [
+                (i, j['core_id'], j['amount']) for j in self.init_data.char_core[i]])
+
+    def insert_purchase_item(self, purchases: list):
+        '''处理singles和packs'''
+        for i in purchases:
+            x = Purchase(self.c).from_dict(i)
+            x.insert_all()
+
+    def item_init(self) -> None:
+        '''初始化物品信息'''
+        self.c.executemany('''insert into item values(?,"core",1)''',
+                           [(i,) for i in self.init_data.cores])
+        self.c.executemany('''insert into item values(?,"world_song",1)''', [
+            (i,) for i in self.init_data.world_songs])
+        self.c.executemany('''insert into item values(?,"world_unlock",1)''', [
+            (i,) for i in self.init_data.world_unlocks])
+        self.c.executemany('''insert into item values(?,"course_banner",1)''', [
+            (i,) for i in self.init_data.course_banners])
+
+        self.c.execute('''insert into item values(?,?,?)''',
+                       ('fragment', 'fragment', 1))
+        self.c.execute('''insert into item values(?,?,?)''',
+                       ('memory', 'memory', 1))
+        self.c.execute('''insert into item values(?,?,?)''',
+                       ('anni5tix', 'anni5tix', 1))
+
+        with open(self.pack_path, 'rb') as f:
+            self.insert_purchase_item(load(f))
+
+        with open(self.single_path, 'rb') as f:
+            self.insert_purchase_item(load(f))
+
+        self.c.execute(
+            '''select exists(select * from item where item_id='epilogue')''')
+        if self.c.fetchone() == (0,):
+            self.c.execute(
+                '''insert into item values('epilogue','pack',1)''')
+
+    def course_init(self) -> None:
+        '''初始化课题信息'''
+        courses = []
+        with open(self.course_path, 'rb') as f:
+            courses = load(f)
+        for i in courses:
+            x = Course(self.c).from_dict(i)
+            x.insert_all()
+
+    def role_power_init(self) -> None:
+        '''初始化power和role'''
+        self.c.executemany('''insert into role values(?,?)''', [(
+            self.init_data.role[i], self.init_data.role_caption[i]) for i in range(len(self.init_data.role))])
+
+        self.c.executemany('''insert into power values(?,?)''', [(
+            self.init_data.power[i], self.init_data.power_caption[i]) for i in range(len(self.init_data.power))])
+
+        for i in self.init_data.role_power:
+            self.c.executemany('''insert into role_power values(?,?)''', [
+                               (i, j) for j in self.init_data.role_power[i]])
+
+    def admin_init(self) -> None:
+        '''初始化测试账号'''
+        x = UserRegister(self.c)
+        x.user_code = '123456789'
+        x.user_id = 2000000
+        x.name = 'admin'
+        x.email = 'admin@admin.com'
+        now = int(time() * 1000)
+
+        x._insert_user_char()
+
+        self.c.execute('''insert into user(user_id, name, password, join_date, user_code, rating_ptt, 
+        character_id, is_skill_sealed, is_char_uncapped, is_char_uncapped_override, is_hide_rating, favorite_character, max_stamina_notification_enabled, current_map, ticket, prog_boost, email)
+        values(:user_id, :name, :password, :join_date, :user_code, 0, 0, 0, 0, 0, 0, -1, 0, '', :memories, 0, :email)
+        ''', {'user_code': x.user_code, 'user_id': x.user_id, 'join_date': now, 'name': x.name, 'password': '41e5653fc7aeb894026d6bb7b2db7f65902b454945fa8fd65a6327047b5277fb', 'memories': 114514, 'email': x.email})
+        self.c.execute('''insert into recent30(user_id) values(:user_id)''', {
+                       'user_id': x.user_id})
+        self.c.execute(
+            '''insert into user_role values(?, "admin")''', (x.user_id,))
+
+    def init(self) -> None:
+        sys.path.append(os.path.join(sys.path[0], self.init_folder_path))
+        self.init_data = import_module('arc_data').InitData
+
+        with Connect(self.db_path) as c:
+            self.c = c
+            self.table_init()
+            self.character_init()
+            self.item_init()
+            self.course_init()
+            self.role_power_init()
+            self.admin_init()
+
+
+class LogDatabaseInit:
+    def __init__(self, db_path: str = Config.SQLITE_LOG_DATABASE_PATH, init_folder_path: str = Config.DATABASE_INIT_PATH) -> None:
+        self.db_path = db_path
+        self.init_folder_path = init_folder_path
+        self.c = None
+
+    @property
+    def sql_path(self) -> str:
+        return os.path.join(self.init_folder_path, 'log_tables.sql')
+
+    def table_init(self) -> None:
+        '''初始化数据库结构'''
+        with open(self.sql_path, 'r') as f:
+            self.c.executescript(f.read())
+        self.c.execute(
+            '''insert into cache values("version", :a, -1);''', {'a': ARCAEA_SERVER_VERSION})
+
+    def init(self) -> None:
+        with Connect(self.db_path) as c:
+            self.c = c
+            self.table_init()
+
+
+class FileChecker:
+    '''文件检查及初始化类'''
+
+    def __init__(self, logger=None):
+        self.logger = logger
+
+    def check_file(self, file_path: str) -> bool:
+        f = os.path.isfile(file_path)
+        if not f:
+            self.logger.warning('File `%s` is missing.' % file_path)
+        return f
+
+    def check_folder(self, folder_path: str) -> bool:
+        f = os.path.isdir(folder_path)
+        if not f:
+            self.logger.warning('Folder `%s` is missing.' % folder_path)
+        return f
+
+    def check_update_database(self) -> bool:
+        if not self.check_file(Config.SQLITE_LOG_DATABASE_PATH):
+            # 新建日志数据库
+            try:
+                self.logger.info(
+                    f'Try to new the file {Config.SQLITE_LOG_DATABASE_PATH}')
+                LogDatabaseInit().init()
+                self.logger.info(
+                    f'Success to new the file {Config.SQLITE_LOG_DATABASE_PATH}')
+            except Exception as e:
+                self.logger.error(format_exc())
+                self.logger.error(
+                    f'Failed to new the file {Config.SQLITE_LOG_DATABASE_PATH}')
+                return False
+        if not self.check_file(Config.SQLITE_DATABASE_PATH):
+            # 新建数据库
+            try:
+                self.logger.info(
+                    'Try to new the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+                DatabaseInit().init()
+                self.logger.info(
+                    'Success to new the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+            except Exception as e:
+                self.logger.error(format_exc())
+                self.logger.warning(
+                    'Failed to new the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+                return False
+        else:
+            # 检查更新
+            with Connect() as c:
+                try:
+                    c.execute('''select value from config where id="version"''')
+                    x = c.fetchone()
+                except:
+                    x = None
+            # 数据库自动更新，不强求
+            if not x or x[0] != ARCAEA_SERVER_VERSION:
+                self.logger.warning(
+                    'Maybe the file `%s` is an old version.' % Config.SQLITE_DATABASE_PATH)
+                try:
+                    self.logger.info(
+                        'Try to update the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+
+                    if not os.path.isdir(Config.SQLITE_DATABASE_BACKUP_FOLDER_PATH):
+                        os.makedirs(Config.SQLITE_DATABASE_BACKUP_FOLDER_PATH)
+
+                    backup_path = try_rename(Config.SQLITE_DATABASE_PATH, os.path.join(
+                        Config.SQLITE_DATABASE_BACKUP_FOLDER_PATH, os.path.split(Config.SQLITE_DATABASE_PATH)[-1] + '.bak'))
+
+                    try:
+                        copy2(backup_path, Config.SQLITE_DATABASE_PATH)
+                    except:
+                        copy(backup_path, Config.SQLITE_DATABASE_PATH)
+
+                    temp_path = os.path.join(
+                        *os.path.split(Config.SQLITE_DATABASE_PATH)[:-1], 'old_arcaea_database.db')
+                    if os.path.isfile(temp_path):
+                        os.remove(temp_path)
+
+                    try_rename(Config.SQLITE_DATABASE_PATH, temp_path)
+
+                    DatabaseInit().init()
+                    self.update_database(temp_path)
+
+                    self.logger.info(
+                        'Success to update the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+
+                except Exception as e:
+                    self.logger.error(format_exc())
+                    self.logger.warning(
+                        'Fail to update the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+
+        return True
+
+    @staticmethod
+    def update_database(old_path: str, new_path: str = Config.SQLITE_DATABASE_PATH) -> None:
+        '''更新数据库，并删除旧文件'''
+        if os.path.isfile(old_path) and os.path.isfile(new_path):
+            DatabaseMigrator(old_path, new_path).update_database()
+            os.remove(old_path)
+
+    def check_song_file(self) -> bool:
+        '''检查song有关文件并初始化缓存'''
+        f = self.check_folder(Config.SONG_FILE_FOLDER_PATH)
+        self.logger.info("Start to initialize song data...")
+        try:
+            DownloadList.initialize_cache()
+            self.logger.info('Complete!')
+        except Exception as e:
+            self.logger.error(format_exc())
+            self.logger.warning('Initialization error!')
+            f = False
+        return f
+
+    def check_before_run(self) -> bool:
+        '''运行前检查，返回布尔值'''
+        MemoryDatabase()  # 初始化内存数据库
+        return self.check_song_file() & self.check_update_database()

latest version/core/item.py

@@ -1,5 +1,5 @@
+from .config_manager import Config
 from .error import InputError, ItemNotEnough, ItemUnavailable, NoData
-from setting import Config
 
 
 class Item:

latest version/core/limiter.py

@@ -0,0 +1,31 @@
+from limits import parse_many, strategies
+from limits.storage import storage_from_string
+
+
+class ArcLimiter:
+    storage = storage_from_string("memory://")
+    strategy = strategies.FixedWindowRateLimiter(storage)
+
+    def __init__(self, limit_str: str = None, namespace: str = None):
+        self._limits: list = None
+        self.limits = limit_str
+        self.namespace = namespace
+
+    @property
+    def limits(self) -> list:
+        return self._limits
+
+    @limits.setter
+    def limits(self, value: str):
+        if value is None:
+            return
+        self._limits = parse_many(value)
+
+    def hit(self, key: str, cost: int = 1) -> bool:
+        flag = True
+        for limit in self.limits:
+            flag &= self.strategy.hit(limit, self.namespace, key, cost)
+        return flag
+
+    def test(self, key: str) -> bool:
+        return all(self.strategy.test(limit, self.namespace, key) for limit in self.limits)

latest version/core/operation.py

@@ -0,0 +1,67 @@
+from .sql import Connect, Sql
+from .score import Score
+from .download import DownloadList
+
+
+class BaseOperation:
+    name: str = None
+
+    def __init__(self):
+        pass
+
+    def __call__(self, *args, **kwargs):
+        return self.run(*args, **kwargs)
+
+    def run(self, *args, **kwargs):
+        raise NotImplementedError
+
+
+class RefreshAllScoreRating(BaseOperation):
+    '''
+        刷新所有成绩的评分
+    '''
+    name = 'refresh_all_score_rating'
+
+    def run(self):
+        # 追求效率，不用Song类，尽量不用对象
+        # 但其实还是很慢
+        with Connect() as c:
+            c.execute(
+                '''select song_id, rating_pst, rating_prs, rating_ftr, rating_byn from chart''')
+            x = c.fetchall()
+
+            songs = [i[0] for i in x]
+            c.execute(
+                f'''update best_score set rating=0 where song_id not in ({','.join(['?']*len(songs))})''', songs)
+
+            for i in x:
+                for j in range(0, 4):
+                    defnum = -10  # 没在库里的全部当做定数-10
+                    if i[j+1] is not None and i[j+1] > 0:
+                        defnum = float(i[j+1]) / 10
+
+                    c.execute('''select user_id, score from best_score where song_id=:a and difficulty=:b''', {
+                              'a': i[0], 'b': j})
+                    y = c.fetchall()
+                    values = []
+                    where_values = []
+                    for k in y:
+                        ptt = Score.calculate_rating(defnum, k[1])
+                        if ptt < 0:
+                            ptt = 0
+                        values.append((ptt,))
+                        where_values.append((k[0], i[0], j))
+                    if values:
+                        Sql(c).update_many('best_score', ['rating'], values, [
+                            'user_id', 'song_id', 'difficulty'], where_values)
+
+
+class RefreshSongFileCache(BaseOperation):
+    '''
+        刷新歌曲文件缓存，包括文件hash缓存重建、文件目录重遍历、songlist重解析
+    '''
+    name = 'refresh_song_file_cache'
+
+    def run(self):
+        DownloadList.clear_all_cache()
+        DownloadList.initialize_cache()

latest version/core/purchase.py

@@ -53,6 +53,36 @@ class Purchase:
                 r['discount_reason'] = self.discount_reason
         return r
 
+    def from_dict(self, d: dict) -> 'Purchase':
+        self.purchase_name = d['name']
+        self.price = d['price']
+        self.orig_price = d['orig_price']
+        self.discount_from = d.get('discount_from', -1)
+        self.discount_to = d.get('discount_to', -1)
+        self.discount_reason = d.get('discount_reason', '')
+        for i in d.get('items', []):
+            self.items.append(ItemFactory.from_dict(i))
+
+        return self
+
+    def insert_all(self) -> None:
+        '''向数据库插入，包括item表和purchase_item表'''
+        self.c.execute('''insert into purchase values(?,?,?,?,?,?)''',
+                       (self.purchase_name, self.price, self.orig_price, self.discount_from, self.discount_to, self.discount_reason))
+        self.insert_items()
+
+    def insert_items(self) -> None:
+        '''向数据库插入物品，注意已存在的物品不会变更'''
+        for i in self.items:
+            self.c.execute(
+                '''select exists(select * from item where item_id=?)''', (i.item_id,))
+            if self.c.fetchone() == (0,):
+                self.c.execute('''insert into item values(?,?,?)''',
+                               (i.item_id, i.item_type, i.is_available))
+
+            self.c.execute('''insert into purchase_item values(?,?,?,?)''',
+                           (self.purchase_name, i.item_id, i.item_type, i.amount))
+
     def select(self, purchase_name: str = None) -> 'Purchase':
         '''
             用purchase_name查询信息

latest version/core/rank.py

@@ -1,6 +1,7 @@
 from .constant import Constant
 from .score import UserScore
 from .song import Chart
+from .sql import Query, Sql
 from .user import UserInfo
 
 
@@ -26,24 +27,29 @@ class RankList:
             得到top分数表
         '''
         if self.limit >= 0:
-            self.c.execute('''select user_id from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC limit :limit''', {
+            self.c.execute('''select * from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC limit :limit''', {
                 'song_id': self.song.song_id, 'difficulty': self.song.difficulty, 'limit': self.limit})
         else:
-            self.c.execute('''select user_id from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC''', {
+            self.c.execute('''select * from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC''', {
                 'song_id': self.song.song_id, 'difficulty': self.song.difficulty})
 
         x = self.c.fetchall()
         if not x:
             return None
 
+        user_info_list = Sql(self.c).select('user', ['user_id', 'name', 'character_id', 'is_skill_sealed', 'is_char_uncapped',
+                                                     'is_char_uncapped_override', 'favorite_character'], Query().from_args({'user_id': [i[0] for i in x]}))
+        user_info_dict = {i[0]: i[1:] for i in user_info_list}
+
         rank = 0
         self.list = []
         for i in x:
             rank += 1
-            y = UserScore(self.c, UserInfo(self.c, i[0]))
+            y = UserScore(self.c, UserInfo(self.c, i[0])).from_list(i)
             y.song = self.song
-            y.select_score()
             y.rank = rank
+            y.user.from_list_about_character(user_info_dict[i[0]])
+
             self.list.append(y)
 
     def select_friend(self, user=None, limit=Constant.MAX_FRIEND_COUNT) -> None:
@@ -54,20 +60,26 @@ class RankList:
         if user:
             self.user = user
 
-        self.c.execute('''select user_id from best_score where user_id in (select :user_id union select user_id_other from friend where user_id_me = :user_id) and song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC limit :limit''', {
-            'user_id': self.user.user_id, 'song_id': self.song.song_id, 'difficulty': self.song.difficulty, 'limit': self.limit})
+        user_ids = [self.user.user_id] + [x[0] for x in self.user.friend_ids]
+
+        self.c.execute(f'''select * from best_score where user_id in ({','.join(['?'] * len(user_ids))}) and song_id = ? and difficulty = ? order by score DESC, time_played DESC limit ?''', user_ids + [
+                       self.song.song_id, self.song.difficulty, self.limit])
         x = self.c.fetchall()
         if not x:
             return None
 
+        user_info_list = Sql(self.c).select('user', ['user_id', 'name', 'character_id', 'is_skill_sealed', 'is_char_uncapped',
+                                                     'is_char_uncapped_override', 'favorite_character'], Query().from_args({'user_id': [i[0] for i in x]}))
+        user_info_dict = {i[0]: i[1:] for i in user_info_list}
         rank = 0
         self.list = []
         for i in x:
             rank += 1
-            y = UserScore(self.c, UserInfo(self.c, i[0]))
+            y = UserScore(self.c, UserInfo(self.c, i[0])).from_list(i)
             y.song = self.song
-            y.select_score()
             y.rank = rank
+            y.user.from_list_about_character(user_info_dict[i[0]])
+
             self.list.append(y)
 
     @staticmethod
@@ -118,19 +130,25 @@ class RankList:
 
         sql_limit, sql_offset, need_myself = self.get_my_rank_parameter(
             my_rank, int(self.c.fetchone()[0]), self.limit)
-        self.c.execute('''select user_id from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC limit :limit offset :offset''', {
+        self.c.execute('''select * from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC limit :limit offset :offset''', {
             'song_id': self.song.song_id, 'difficulty': self.song.difficulty, 'limit': sql_limit, 'offset': sql_offset})
         x = self.c.fetchall()
+
         if x:
+            user_info_list = Sql(self.c).select('user', ['user_id', 'name', 'character_id', 'is_skill_sealed', 'is_char_uncapped',
+                                                         'is_char_uncapped_override', 'favorite_character'], Query().from_args({'user_id': [i[0] for i in x]}))
+            user_info_dict = {i[0]: i[1:] for i in user_info_list}
             rank = sql_offset if sql_offset > 0 else 0
             self.list = []
             for i in x:
                 rank += 1
-                y = UserScore(self.c, UserInfo(self.c, i[0]))
+                y = UserScore(self.c, UserInfo(self.c, i[0])).from_list(i)
                 y.song = self.song
-                y.select_score()
                 y.rank = rank
+                y.user.from_list_about_character(user_info_dict[i[0]])
+
                 self.list.append(y)
+
             if need_myself:
                 y = UserScore(self.c, UserInfo(self.c, self.user.user_id))
                 y.song = self.song

latest version/core/save.py

@@ -1,10 +1,8 @@
 import json
 from time import time
 
-from setting import Config
-
-from core.constant import Constant
-
+from .config_manager import Config
+from .constant import Constant
 from .error import InputError
 from .util import md5
 
@@ -101,9 +99,7 @@ class SaveData:
             parameter: `user` - `User`类或子类的实例
         '''
         self.createdAt = int(time() * 1000)
-        self.c.execute('''delete from user_save where user_id=:a''', {
-                       'a': user.user_id})
-        self.c.execute('''insert into user_save values(:a,:b,:c,:d,:e,:f,:g,:h,:i,:j)''', {
+        self.c.execute('''insert or replace into user_save values(:a,:b,:c,:d,:e,:f,:g,:h,:i,:j)''', {
             'a': user.user_id, 'b': json.dumps({'': self.scores_data}), 'c': json.dumps({'': self.clearlamps_data}), 'd': json.dumps({'': self.clearedsongs_data}), 'e': json.dumps({'': self.unlocklist_data}), 'f': json.dumps({'val': self.installid_data}), 'g': json.dumps({'val': self.devicemodelname_data}), 'h': json.dumps({'': self.story_data}), 'i': self.createdAt, 'j': self.finalestate_data})
 
     def set_value(self, key: str, value: str, checksum: str) -> None:

latest version/core/score.py

@@ -7,7 +7,7 @@ from .course import CoursePlay
 from .error import NoData, StaminaNotEnough
 from .item import ItemCore
 from .song import Chart
-from .sql import Query, Sql
+from .sql import Connect, Query, Sql
 from .util import md5
 from .world import WorldPlay
 
@@ -103,7 +103,7 @@ class Score:
 
     @staticmethod
     def calculate_rating(defnum: int, score: int) -> float:
-        '''计算rating，谱面定数小于等于0视为Unrank，这里的defnum = Chart const'''
+        '''计算rating，谱面定数小于等于0视为Unrank，返回值会为-1，这里的defnum = Chart const'''
         if not defnum or defnum <= 0:
             # 谱面没定数或者定数小于等于0被视作Unrank
             return -1
@@ -155,9 +155,10 @@ class UserScore(Score):
         super().__init__()
         self.c = c
         self.user = user
-        self.rank = None
+        self.rank = None  # 成绩排名，给Ranklist用的
 
     def select_score(self) -> None:
+        '''查询成绩以及用户搭档信息，单次查询可用，不要集体循环查询'''
         self.c.execute('''select * from best_score where user_id = :a and song_id = :b and difficulty = :c''',
                        {'a': self.user.user_id, 'b': self.song.song_id, 'c': self.song.difficulty})
         x = self.c.fetchone()
@@ -258,6 +259,12 @@ class UserPlay(UserScore):
 
     def get_play_state(self) -> None:
         '''检查token，当然这里不管有没有，是用来判断世界模式和课题模式的'''
+        if self.song_token == '1145141919810':
+            # 硬编码检查，绕过数据库
+            self.is_world_mode = False
+            self.course_play_state = -1
+            return None
+
         self.c.execute(
             '''select * from songplay_token where token=:a ''', {'a': self.song_token})
         x = self.c.fetchone()
@@ -265,7 +272,8 @@ class UserPlay(UserScore):
             self.is_world_mode = False
             self.course_play_state = -1
             return None
-        self.song.set_chart(x[2], x[3])
+            # raise NoData('No token data.')
+        # self.song.set_chart(x[2], x[3])
         if x[4]:
             self.course_play = CoursePlay(self.c, self.user, self)
             self.course_play.course_id = x[4]
@@ -375,12 +383,12 @@ class UserPlay(UserScore):
                 r30_id = 29
             elif self.song.song_id_difficulty in songs and n == 10:
                 i = 29
-                while self.ptt.s30[i] == self.song.song_id_difficulty:
+                while self.ptt.s30[i] == self.song.song_id_difficulty and i > 0:
                     i -= 1
                 r30_id = i
             elif self.song.song_id_difficulty not in songs and n == 9:
                 i = 29
-                while self.ptt.s30.count(self.ptt.s30[-1]) == 1:
+                while self.ptt.s30.count(self.ptt.s30[i]) == 1 and i > 0:
                     i -= 1
                 r30_id = i
             else:
@@ -400,6 +408,12 @@ class UserPlay(UserScore):
 
         self.ptt.insert_recent_30()
 
+    def record_score(self) -> None:
+        '''向log数据库记录分数，请注意列名不同'''
+        with Connect(Constant.SQLITE_LOG_DATABASE_PATH) as c2:
+            c2.execute('''insert into user_score values(?,?,?,?,?,?,?,?,?,?,?,?,?)''', (self.user.user_id, self.song.song_id, self.song.difficulty, self.time_played,
+                                                                                        self.score, self.shiny_perfect_count, self.perfect_count, self.near_count, self.miss_count, self.health, self.modifier, self.clear_type, self.rating))
+
     def upload_score(self) -> None:
         '''上传分数，包括user的recent更新，best更新，recent30更新，世界模式计算'''
         self.get_play_state()
@@ -412,6 +426,9 @@ class UserPlay(UserScore):
 
         self.time_played = int(time())
 
+        # 记录分数
+        self.record_score()
+
         # recent更新
         self.c.execute('''update user set song_id = :b, difficulty = :c, score = :d, shiny_perfect_count = :e, perfect_count = :f, near_count = :g, miss_count = :h, health = :i, modifier = :j, clear_type = :k, rating = :l, time_played = :m  where user_id = :a''', {
             'a': self.user.user_id, 'b': self.song.song_id, 'c': self.song.difficulty, 'd': self.score, 'e': self.shiny_perfect_count, 'f': self.perfect_count, 'g': self.near_count, 'h': self.miss_count, 'i': self.health, 'j': self.modifier, 'k': self.clear_type, 'l': self.rating, 'm': self.time_played * 1000})
@@ -487,6 +504,9 @@ class Potential:
         self.c.execute(
             '''select * from recent30 where user_id = :a''', {'a': self.user.user_id})
         x = self.c.fetchone()
+        if not x:
+            raise NoData(
+                f'No recent30 data for user `{self.user.user_id}`', api_error_code=-3)
         self.r30 = []
         self.s30 = []
         if not x:
@@ -521,11 +541,15 @@ class Potential:
     def recent_30_to_dict_list(self) -> list:
         if self.r30 is None:
             self.select_recent_30()
-        return [{
-            'song_id': self.s30[i][:-1],
-            'difficulty': int(self.s30[i][-1]),
-            'rating': self.r30[i]
-        } for i in range(len(self.r30))]
+        r = []
+        for x, y in zip(self.s30, self.r30):
+            if x:
+                r.append({
+                    'song_id': x[:-1],
+                    'difficulty': int(x[-1]),
+                    'rating': y
+                })
+        return r
 
     def recent_30_update(self, pop_index: int, rating: float, song_id_difficulty: str) -> None:
         self.r30.pop(pop_index)
@@ -580,4 +604,5 @@ class UserScoreList:
         for score in self.scores:
             self.c.execute(
                 '''select name from chart where song_id = ?''', (score.song.song_id,))
-            score.song.song_name = self.c.fetchone()[0]
+            x = self.c.fetchone()
+            score.song.song_name = x[0] if x else ''

latest version/core/song.py

@@ -1,4 +1,5 @@
 from .error import NoData
+from .config_manager import Config
 
 
 class Chart:
@@ -33,8 +34,10 @@ class Chart:
             '''select rating_pst, rating_prs, rating_ftr, rating_byn from chart where song_id=:a''', {'a': self.song_id})
         x = self.c.fetchone()
         if x is None:
-            self.defnum = -10
-            # raise NoData('The song `%s` does not exist.' % self.song_id)
+            if Config.ALLOW_SCORE_WITH_NO_SONG:
+                self.defnum = -10
+            else:
+                raise NoData(f'The song `{self.song_id}` does not exist.', 120)
         else:
             self.defnum = x[self.difficulty]
 

latest version/core/sql.py

@@ -1,7 +1,6 @@
 import sqlite3
 import traceback
-
-from flask import current_app
+from atexit import register
 
 from .constant import Constant
 from .error import ArcError, InputError
@@ -9,17 +8,25 @@ from .error import ArcError, InputError
 
 class Connect:
     # 数据库连接类，上下文管理
+    logger = None
 
-    def __init__(self, file_path=Constant.SQLITE_DATABASE_PATH):
+    def __init__(self, file_path: str = Constant.SQLITE_DATABASE_PATH, in_memory: bool = False, logger=None) -> None:
         """
             数据库连接，默认连接arcaea_database.db\ 
             接受：文件路径\ 
             返回：sqlite3连接操作对象
         """
         self.file_path = file_path
+        self.in_memory: bool = in_memory
+        if logger is not None:
+            self.logger = logger
 
-    def __enter__(self):
-        self.conn = sqlite3.connect(self.file_path, timeout=10)
+    def __enter__(self) -> sqlite3.Cursor:
+        if self.in_memory:
+            self.conn = sqlite3.connect(
+                'file:arc_tmp?mode=memory&cache=shared', uri=True, timeout=10)
+        else:
+            self.conn = sqlite3.connect(self.file_path, timeout=10)
         self.c = self.conn.cursor()
         return self.c
 
@@ -29,15 +36,13 @@ class Connect:
             if issubclass(exc_type, ArcError):
                 flag = False
             else:
-                if self.conn:
-                    self.conn.rollback()
+                self.conn.rollback()
 
-                current_app.logger.error(
+                self.logger.error(
                     traceback.format_exception(exc_type, exc_val, exc_tb))
 
-        if self.conn:
-            self.conn.commit()
-            self.conn.close()
+        self.conn.commit()
+        self.conn.close()
 
         return flag
 
@@ -45,15 +50,18 @@ class Connect:
 class Query:
     '''查询参数类'''
 
-    def __init__(self, query_able: list = None, quzzy_query_able: list = None, sort_able: list = None) -> None:
-        self.query_able: list = query_able
-        self.quzzy_query_able: list = quzzy_query_able
+    def __init__(self, query_able: list = None, fuzzy_query_able: list = None, sort_able: list = None) -> None:
+        self.query_able: list = query_able  # None表示不限制
+        self.fuzzy_query_able: list = fuzzy_query_able  # None表示不限制
         self.sort_able: list = sort_able
 
         self.__limit: int = -1
         self.__offset: int = 0
-        self.__query: dict = {}  # {'name': 'admin'}
+
+        # {'name': 'admin'} or {'name': ['admin', 'user']}
+        self.__query: dict = {}
         self.__fuzzy_query: dict = {}  # {'name': 'dmi'}
+
         # [{'column': 'user_id', 'order': 'ASC'}, ...]
         self.__sort: list = []
 
@@ -108,7 +116,7 @@ class Query:
     def fuzzy_query_append(self, fuzzy_query: dict) -> None:
         if not isinstance(fuzzy_query, dict):
             raise InputError(api_error_code=-101)
-        if self.quzzy_query_able is not None and fuzzy_query and not set(fuzzy_query).issubset(set(self.quzzy_query_able)):
+        if self.fuzzy_query_able is not None and fuzzy_query and not set(fuzzy_query).issubset(set(self.fuzzy_query_able)):
             raise InputError(api_error_code=-102)
         if not self.__fuzzy_query:
             self.__fuzzy_query = fuzzy_query
@@ -143,11 +151,15 @@ class Query:
         self.fuzzy_query = fuzzy_query
         self.sort = sort
 
-    def from_data(self, d: dict) -> 'Query':
+    def from_dict(self, d: dict) -> 'Query':
         self.set_value(d.get('limit', -1), d.get('offset', 0),
                        d.get('query', {}), d.get('fuzzy_query', {}), d.get('sort', []))
         return self
 
+    def from_args(self, query: dict, limit: int = -1, offset: int = 0, sort: list = [], fuzzy_query: dict = {}) -> 'Query':
+        self.set_value(limit, offset, query, fuzzy_query, sort)
+        return self
+
 
 class Sql:
     '''
@@ -157,63 +169,256 @@ class Sql:
     def __init__(self, c=None) -> None:
         self.c = c
 
-    def select(self, table_name: str, target_column: 'list' = [], query: 'Query' = None) -> list:
-        '''单表内行查询单句sql语句，返回fetchall数据'''
-
-        sql = 'select '
+    @staticmethod
+    def get_select_sql(table_name: str, target_column: list = [], query: 'Query' = None):
+        '''拼接单表内行查询单句sql语句，返回语句和参数列表'''
         sql_list = []
-        if len(target_column) >= 2:
-            sql += target_column[0]
-            for i in range(1, len(target_column)):
-                sql += ',' + target_column[i]
-            sql += ' from ' + table_name
-        elif len(target_column) == 1:
-            sql += target_column[0] + ' from ' + table_name
+        if not target_column:
+            sql = f'select * from {table_name}'
         else:
-            sql += '* from ' + table_name
+            sql = f"select {', '.join(target_column)} from {table_name}"
 
         if query is None:
-            self.c.execute(sql)
-            return self.c.fetchall()
+            return sql, sql_list
 
         where_key = []
-        where_like_key = []
-        for i in query.query:
-            where_key.append(i)
-            sql_list.append(query.query[i])
-
-        for i in query.fuzzy_query:
-            where_like_key.append(i)
-            sql_list.append('%' + query.fuzzy_query[i] + '%')
-
-        if where_key or where_like_key:
-            sql += ' where '
-            if where_key:
-                sql += where_key[0] + '=?'
-                if len(where_key) >= 2:
-                    for i in range(1, len(where_key)):
-                        sql += ' and ' + where_key[i] + '=?'
-                if where_like_key:
-                    for i in where_like_key:
-                        sql += ' and ' + i + ' like ?'
+        for k, v in query.query.items():
+            if isinstance(v, list):
+                where_key.append(f"{k} in ({','.join(['?'] * len(v))})")
+                sql_list.extend(v)
             else:
-                sql += where_like_key[0] + ' like ?'
+                where_key.append(f'{k}=?')
+                sql_list.append(v)
 
-                if len(where_like_key) >= 2:
-                    for i in range(1, len(where_key)):
-                        sql += ' and ' + where_key[i] + ' like ?'
+        for k, v in query.fuzzy_query.items():
+            where_key.append(f'{k} like ?')
+            sql_list.append(f'%{v}%')
+
+        if where_key:
+            sql += ' where '
+            sql += ' and '.join(where_key)
 
         if query.sort:
             sql += ' order by ' + \
-                query.sort[0]['column'] + ' ' + query.sort[0]['order']
-            for i in range(1, len(query.sort)):
-                sql += ', ' + query.sort[i]['column'] + \
-                    ' ' + query.sort[i]['order']
+                ', '.join([x['column'] + ' ' + x['order'] for x in query.sort])
 
         if query.limit >= 0:
             sql += ' limit ? offset ?'
             sql_list.append(query.limit)
             sql_list.append(query.offset)
 
+        return sql, sql_list
+
+    @staticmethod
+    def get_insert_sql(table_name: str, key: list = [], value_len: int = None, insert_type: str = None) -> str:
+        '''拼接insert语句，请注意只返回sql语句，insert_type为replace或ignore'''
+        insert_type = 'replace' if insert_type in [
+            'replace', 'R', 'r', 'REPLACE'] else 'ignore'
+        return ('insert into ' if insert_type is None else 'insert or ' + insert_type + ' into ') + table_name + ('(' + ','.join(key) + ')' if key else '') + ' values(' + ','.join(['?'] * (len(key) if value_len is None else value_len)) + ')'
+
+    @staticmethod
+    def get_update_sql(table_name: str, d: dict = None, query: 'Query' = None):
+        if not d:
+            return None
+        sql_list = []
+        sql = f"update {table_name} set {','.join([f'{k}=?' for k in d.keys()])}"
+        sql_list.extend(d.values())
+
+        if query is None:
+            return sql, sql_list
+
+        where_key = []
+        for k, v in query.query.items():
+            if isinstance(v, list):
+                where_key.append(f"{k} in ({','.join(['?'] * len(v))})")
+                sql_list.extend(v)
+            else:
+                where_key.append(f'{k}=?')
+                sql_list.append(v)
+
+        for k, v in query.fuzzy_query.items():
+            where_key.append(f'{k} like ?')
+            sql_list.append(f'%{v}%')
+
+        if where_key:
+            sql += ' where '
+            sql += ' and '.join(where_key)
+
+        return sql, sql_list
+
+    @staticmethod
+    def get_update_many_sql(table_name: str, key: list = None, where_key: list = None) -> str:
+        '''拼接update语句，这里不用Query类，也不用字典，请注意只返回sql语句'''
+        if not key or not where_key:
+            return None
+        return f"update {table_name} set {','.join([f'{k}=?' for k in key])} where {' and '.join([f'{k}=?' for k in where_key])}"
+
+    @staticmethod
+    def get_delete_sql(table_name: str, query: 'Query' = None):
+        '''拼接删除语句，query中只有query和fuzzy_query会被处理'''
+        sql = f'delete from {table_name}'
+
+        if query is None:
+            return sql, []
+
+        sql_list = []
+        where_key = []
+        for k, v in query.query.items():
+            if isinstance(v, list):
+                where_key.append(f"{k} in ({','.join(['?'] * len(v))})")
+                sql_list.extend(v)
+            else:
+                where_key.append(f'{k}=?')
+                sql_list.append(v)
+
+        for k, v in query.fuzzy_query.items():
+            where_key.append(f'{k} like ?')
+            sql_list.append(f'%{v}%')
+
+        if where_key:
+            sql += ' where '
+            sql += ' and '.join(where_key)
+
+        return sql, sql_list
+
+    def select(self, table_name: str, target_column: list = [], query: 'Query' = None) -> list:
+        '''单表内行select单句sql语句，返回fetchall数据'''
+        sql, sql_list = self.get_select_sql(table_name, target_column, query)
         self.c.execute(sql, sql_list)
         return self.c.fetchall()
+
+    def select_exists(self, table_name: str, target_column: list = [], query: 'Query' = None) -> bool:
+        '''单表内行select exists单句sql语句，返回bool值'''
+        sql, sql_list = self.get_select_sql(table_name, target_column, query)
+        self.c.execute('select exists(' + sql + ')', sql_list)
+        return self.c.fetchone() == (1,)
+
+    def insert(self, table_name: str, key: list, value: tuple, insert_type: str = None) -> None:
+        '''单行插入或覆盖插入，key传[]则为全部列，insert_type为replace或ignore'''
+        self.c.execute(self.get_insert_sql(
+            table_name, key, len(value), insert_type), value)
+
+    def insert_many(self, table_name: str, key: list, value_list: list, insert_type: str = None) -> None:
+        '''多行插入或覆盖插入，key传[]则为全部列，insert_type为replace或ignore'''
+        if not value_list:
+            return
+        self.c.executemany(self.get_insert_sql(
+            table_name, key, len(value_list[0]), insert_type), value_list)
+
+    def update(self, table_name: str, d: dict, query: 'Query' = None) -> None:
+        '''单表内行update单句sql语句'''
+        if not d:
+            return
+        sql, sql_list = self.get_update_sql(table_name, d, query)
+        self.c.execute(sql, sql_list)
+
+    def update_many(self, table_name: str, key: list, value_list: list, where_key: list, where_value_list: list) -> None:
+        '''单表内行update多句sql语句，这里不用Query类，也不用字典，要求值list长度一致，有点像insert_many'''
+        if not key or not value_list or not where_key or not where_value_list or not len(key) == len(value_list[0]) or not len(where_key) == len(where_value_list[0]) or not len(value_list) == len(where_value_list):
+            raise ValueError
+        self.c.executemany(self.get_update_many_sql(
+            table_name, key, where_key), [x + y for x, y in zip(value_list, where_value_list)])
+
+    def delete(self, table_name: str, query: 'Query' = None) -> None:
+        '''删除，query中只有query和fuzzy_query会被处理'''
+        sql, sql_list = self.get_delete_sql(table_name, query)
+        self.c.execute(sql, sql_list)
+
+    def get_table_info(self, table_name: str):
+        '''得到表结构，返回主键列表和字段名列表'''
+        pk = []
+        name = []
+
+        self.c.execute('''pragma table_info ("%s")''' % table_name)  # 这里无法参数化
+        x = self.c.fetchall()
+        if x:
+            for i in x:
+                name.append(i[1])
+                if i[5] != 0:
+                    pk.append(i[1])
+
+        return pk, name
+
+
+class DatabaseMigrator:
+
+    def __init__(self, c1_path: str, c2_path: str) -> None:
+        self.c1_path = c1_path
+        self.c2_path = c2_path
+
+    @staticmethod
+    def update_one_table(c1, c2, table_name: str) -> bool:
+        '''从c1向c2更新数据表，c1中存在的信息不变，即c2中的冲突信息会被覆盖'''
+        c1.execute(
+            '''select * from sqlite_master where type = 'table' and name = :a''', {'a': table_name})
+        c2.execute(
+            '''select * from sqlite_master where type = 'table' and name = :a''', {'a': table_name})
+        if not c1.fetchone() or not c2.fetchone():
+            return False
+
+        sql1 = Sql(c1)
+        sql2 = Sql(c2)
+        db1_pk, db1_name = sql1.get_table_info(table_name)
+        db2_pk, db2_name = sql2.get_table_info(table_name)
+        if db1_pk != db2_pk:
+            return False
+
+        sql2.insert_many(table_name, [], sql1.select(
+            table_name, list(filter(lambda x: x in db2_name, db1_name))), insert_type='replace')
+
+        return True
+
+    @staticmethod
+    def update_user_char_full(c) -> None:
+        '''用character表数据更新user_char_full'''
+        c.execute('''select character_id, max_level, is_uncapped from character''')
+        x = c.fetchall()
+        c.execute('''select user_id from user''')
+        y = c.fetchall()
+        c.execute('''delete from user_char_full''')
+        for i in x:
+            exp = 25000 if i[1] == 30 else 10000
+            c.executemany('''insert into user_char_full values(?,?,?,?,?,?)''', [
+                          (j[0], i[0], i[1], exp, i[2], 0) for j in y])
+
+    @staticmethod
+    def update_user_epilogue(c) -> None:
+        '''给用户添加epilogue包'''
+        c.execute('''select user_id from user''')
+        Sql(c).insert_many('user_item', [], [(i[0], 'epilogue', 'pack', 1)
+                                             for i in c.fetchall()], insert_type='ignore')
+
+    def update_database(self) -> None:
+        '''
+        将c1数据库不存在数据加入或覆盖到c2数据库上
+        对于c2，更新一些表，并用character数据更新user_char_full
+        '''
+        with Connect(self.c2_path) as c2:
+            with Connect(self.c1_path) as c1:
+                [self.update_one_table(c1, c2, i)
+                 for i in Constant.DATABASE_MIGRATE_TABLES]
+
+                if not Constant.UPDATE_WITH_NEW_CHARACTER_DATA:
+                    self.update_one_table(c1, c2, 'character')
+
+            self.update_user_char_full(c2)  # 更新user_char_full
+            self.update_user_epilogue(c2)  # 更新user的epilogue
+
+
+class MemoryDatabase:
+    conn = sqlite3.connect('file:arc_tmp?mode=memory&cache=shared', uri=True)
+
+    def __init__(self):
+        self.c = self.conn.cursor()
+        self.c.execute('''PRAGMA journal_mode = OFF''')
+        self.c.execute('''PRAGMA synchronous = 0''')
+        self.c.execute('''create table if not exists download_token(user_id int, 
+        song_id text,file_name text,token text,time int,primary key(user_id, song_id, file_name));''')
+        self.c.execute(
+            '''create index if not exists download_token_1 on download_token (song_id, file_name);''')
+        self.conn.commit()
+
+
+@register
+def atexit():
+    MemoryDatabase.conn.close()

latest version/core/user.py

@@ -3,15 +3,15 @@ import hashlib
 import time
 from os import urandom
 
-from setting import Config
-
 from .character import UserCharacter, UserCharacterList
+from .config_manager import Config
 from .constant import Constant
 from .error import (ArcError, DataExist, FriendError, InputError, NoAccess,
-                    NoData, UserBan)
+                    NoData, RateLimit, UserBan)
 from .item import UserItemList
+from .limiter import ArcLimiter
 from .score import Score
-from .sql import Connect
+from .sql import Query, Sql
 from .world import Map, UserMap, UserStamina
 
 
@@ -32,17 +32,17 @@ def code_get_id(c, user_code: str) -> int:
 
 class User:
     def __init__(self) -> None:
-        self.name = None
-        self.email = None
-        self.password = None
-        self.user_id = None
-        self.user_code = None
+        self.name: str = None
+        self.email: str = None
+        self.password: str = None
+        self.user_id: int = None
+        self.user_code: str = None
 
         self.join_date = None
         self.rating_ptt: int = None  # 100 times
 
-        self.ticket = None
-        self.world_rank_score = None
+        self.ticket: int = None
+        self.world_rank_score: int = None
         self.ban_flag = None
 
     @property
@@ -63,7 +63,7 @@ class UserRegister(User):
             if self.c.fetchone() == (0,):
                 self.name = name
             else:
-                raise DataExist('Username exists.', 101, -203)
+                raise DataExist('Username exists.', 101, -210)
 
         else:
             raise InputError('Username is invalid.')
@@ -76,16 +76,28 @@ class UserRegister(User):
 
     def set_email(self, email: str):
         # 邮箱格式懒得多判断
-        if 4 <= len(email) <= 32 and '@' in email and '.' in email:
+        if 4 <= len(email) <= 64 and '@' in email and '.' in email:
             self.c.execute(
                 '''select exists(select * from user where email = :email)''', {'email': email})
             if self.c.fetchone() == (0,):
                 self.email = email
             else:
-                raise DataExist('Email address exists.', 102, -204)
+                raise DataExist('Email address exists.', 102, -211)
         else:
             raise InputError('Email address is invalid.')
 
+    def set_user_code(self, user_code: str) -> None:
+        '''设置用户的user_code'''
+        if len(user_code) == 9 and user_code.isdigit():
+            self.c.execute(
+                '''select exists(select * from user where user_code = ?)''', (user_code, ))
+            if self.c.fetchone() == (0,):
+                self.user_code = user_code
+            else:
+                raise DataExist('User code exists.', 103, -212)
+        else:
+            raise InputError('User code is invalid.')
+
     def _build_user_code(self):
         # 生成9位的user_code，用的自然是随机
         from random import randint
@@ -130,8 +142,10 @@ class UserRegister(User):
 
     def register(self):
         now = int(time.time() * 1000)
-        self._build_user_code()
-        self._build_user_id()
+        if self.user_code is None:
+            self._build_user_code()
+        if self.user_id is None:
+            self._build_user_id()
         self._insert_user_char()
 
         self.c.execute('''insert into user(user_id, name, password, join_date, user_code, rating_ptt, 
@@ -144,6 +158,8 @@ class UserRegister(User):
 
 class UserLogin(User):
     # 密码和token的加密方式为 SHA-256
+    limiter = ArcLimiter(Config.GAME_LOGIN_RATE_LIMIT, 'game_login')
+
     def __init__(self, c) -> None:
         super().__init__()
         self.c = c
@@ -220,6 +236,9 @@ class UserLogin(User):
         if ip:
             self.set_ip(ip)
 
+        if not self.limiter.hit(name):
+            raise RateLimit('Too many login attempts.', 123, -203)
+
         self.c.execute('''select user_id, password, ban_flag from user where name = :name''', {
                        'name': self.name})
         x = self.c.fetchone()
@@ -276,7 +295,7 @@ class UserAuth(User):
 
 class UserInfo(User):
     def __init__(self, c, user_id=None) -> None:
-        super().__init__()
+        User.__init__(self)
         self.c = c
         self.user_id = user_id
         self.character = None
@@ -370,40 +389,42 @@ class UserInfo(User):
         self.favorite_character.select_character_uncap_condition(self)
         return self.favorite_character
 
+    @property
+    def friend_ids(self) -> list:
+        self.c.execute('''select user_id_other from friend where user_id_me = :user_id''', {
+            'user_id': self.user_id})
+        return self.c.fetchall()
+
     @property
     def friends(self) -> list:
         # 得到用户的朋友列表
         if self.__friends is None:
-            self.c.execute('''select user_id_other from friend where user_id_me = :user_id''', {
-                'user_id': self.user_id})
-            x = self.c.fetchall()
             s = []
-            if x != [] and x[0][0] is not None:
-                for i in x:
-                    self.c.execute('''select exists(select * from friend where user_id_me = :x and user_id_other = :y)''',
-                                   {'x': i[0], 'y': self.user_id})
+            for i in self.friend_ids:
+                self.c.execute('''select exists(select * from friend where user_id_me = :x and user_id_other = :y)''',
+                               {'x': i[0], 'y': self.user_id})
 
-                    is_mutual = True if self.c.fetchone() == (1,) else False
+                is_mutual = True if self.c.fetchone() == (1,) else False
 
-                    you = UserOnline(self.c, i[0])
-                    you.select_user()
-                    character = you.character if you.favorite_character is None else you.favorite_character
-                    character.select_character_uncap_condition(you)
+                you = UserOnline(self.c, i[0])
+                you.select_user()
+                character = you.character if you.favorite_character is None else you.favorite_character
+                character.select_character_uncap_condition(you)
 
-                    rating = you.rating_ptt if not you.is_hide_rating else -1
+                rating = you.rating_ptt if not you.is_hide_rating else -1
 
-                    s.append({
-                        "is_mutual": is_mutual,
-                        "is_char_uncapped_override": character.is_uncapped_override,
-                        "is_char_uncapped": character.is_uncapped,
-                        "is_skill_sealed": you.is_skill_sealed,
-                        "rating": rating,
-                        "join_date": you.join_date,
-                        "character": character.character_id,
-                        "recent_score": you.recent_score_list,
-                        "name": you.name,
-                        "user_id": you.user_id
-                    })
+                s.append({
+                    "is_mutual": is_mutual,
+                    "is_char_uncapped_override": character.is_uncapped_override,
+                    "is_char_uncapped": character.is_uncapped,
+                    "is_skill_sealed": you.is_skill_sealed,
+                    "rating": rating,
+                    "join_date": you.join_date,
+                    "character": character.character_id,
+                    "recent_score": you.recent_score_list,
+                    "name": you.name,
+                    "user_id": you.user_id
+                })
             s.sort(key=lambda item: item["recent_score"][0]["time_played"] if len(
                 item["recent_score"]) > 0 else 0, reverse=True)
             self.__friends = s
@@ -561,16 +582,8 @@ class UserInfo(User):
         self.stamina = UserStamina(self.c, self)
         self.stamina.set_value(x[0], x[1])
 
-    def select_user_about_character(self) -> None:
-        '''
-            查询user表有关角色的信息
-        '''
-        self.c.execute('''select name, character_id, is_skill_sealed, is_char_uncapped, is_char_uncapped_override, favorite_character from user where user_id = :a''', {
-            'a': self.user_id})
-        x = self.c.fetchone()
-        if not x:
-            raise NoData('No user.', 108, -3)
-
+    def from_list_about_character(self, x: list) -> None:
+        '''从数据库user表获取搭档信息'''
         self.name = x[0]
         self.character = UserCharacter(self.c, x[1], self)
         self.is_skill_sealed = x[2] == 1
@@ -579,6 +592,18 @@ class UserInfo(User):
         self.favorite_character = None if x[5] == - \
             1 else UserCharacter(self.c, x[5], self)
 
+    def select_user_about_character(self) -> None:
+        '''
+            查询user表有关搭档的信息
+        '''
+        self.c.execute('''select name, character_id, is_skill_sealed, is_char_uncapped, is_char_uncapped_override, favorite_character from user where user_id = :a''', {
+            'a': self.user_id})
+        x = self.c.fetchone()
+        if not x:
+            raise NoData('No user.', 108, -3)
+
+        self.from_list_about_character(x)
+
     def select_user_about_world_play(self) -> None:
         '''
             查询user表有关世界模式打歌的信息
@@ -616,9 +641,9 @@ class UserInfo(User):
 
     def update_global_rank(self) -> None:
         '''用户世界排名计算，有新增成绩则要更新'''
-        with Connect() as c2:
-            c2.execute('''select song_id, rating_ftr, rating_byn from chart''')
-            x = c2.fetchall()
+
+        self.c.execute('''select song_id, rating_ftr, rating_byn from chart''')
+        x = self.c.fetchall()
 
         song_list_ftr = [self.user_id]
         song_list_byn = [self.user_id]
@@ -720,3 +745,21 @@ class UserOnline(UserInfo):
         self.favorite_character = UserCharacter(self.c, character_id, self)
         self.c.execute('''update user set favorite_character = :a where user_id = :b''',
                        {'a': self.favorite_character.character_id, 'b': self.user_id})
+
+
+class UserChanger(UserInfo, UserRegister):
+
+    def __init__(self, c, user_id=None) -> None:
+        super().__init__(c, user_id)
+
+    def update_columns(self, columns: list = None, d: dict = None) -> None:
+        if columns is not None:
+            d = {}
+            for column in columns:
+                if column == 'password':
+                    d[column] = self.hash_pwd
+                else:
+                    d[column] = self.__dict__[column]
+
+        Sql(self.c).update('user', d, Query().from_args(
+            {'user_id': self.user_id}))

latest version/core/util.py

@@ -2,7 +2,7 @@ import hashlib
 import os
 
 
-def md5(code):
+def md5(code: str) -> str:
     # md5加密算法
     code = code.encode()
     md5s = hashlib.md5()
@@ -12,8 +12,8 @@ def md5(code):
     return codes
 
 
-def get_file_md5(file_path):
-    # 计算文件MD5
+def get_file_md5(file_path: str) -> str:
+    '''计算文件MD5'''
     if not os.path.isfile(file_path):
         return None
     myhash = hashlib.md5()
@@ -25,3 +25,17 @@ def get_file_md5(file_path):
             myhash.update(b)
 
     return myhash.hexdigest()
+
+
+def try_rename(path: str, new_path: str) -> str:
+    '''尝试重命名文件，并尝试避免命名冲突（在后面加自增数字），返回最终路径'''
+    final_path = new_path
+    if os.path.exists(new_path):
+        i = 1
+        while os.path.exists(new_path + str(i)):
+            i += 1
+
+        final_path = new_path + str(i)
+
+    os.rename(path, final_path)
+    return final_path

latest version/core/world.py

@@ -477,7 +477,7 @@ class WorldPlay:
 
         if self.character_bonus_progress is not None:
             # 猜的，为了让客户端正确显示，当然结果是没问题的
-            r['base_progress'] += self.character_bonus_progress
+            # r['base_progress'] += self.character_bonus_progress # 肯定不是这样的
             r['character_bonus_progress'] = self.character_bonus_progress
 
         if self.user_play.beyond_gauge == 0:
@@ -593,7 +593,7 @@ class WorldPlay:
 
     def after_climb(self) -> None:
         factory_dict = {'eto_uncap': self._eto_uncap, 'ayu_uncap': self._ayu_uncap,
-                        'luna_uncap': self._luna_uncap, 'skill_fatalis': self._skill_fatalis}
+                        'luna_uncap': self._luna_uncap, 'skill_fatalis': self._skill_fatalis, 'skill_amane': self._skill_amane}
         if self.character_used.skill_id_displayed in factory_dict:
             factory_dict[self.character_used.skill_id_displayed]()
 
@@ -666,3 +666,14 @@ class WorldPlay:
         self.user.world_mode_locked_end_ts = int(
             time()*1000) + Constant.SKILL_FATALIS_WORLD_LOCKED_TIME
         self.user.update_user_one_column('world_mode_locked_end_ts')
+
+    def _skill_amane(self) -> None:
+        '''
+        amane技能，起始格为限速或随机，成绩小于EX时，世界模式进度减半
+        偷懒在after_climb里面，需要重爬一次
+        '''
+        x: 'Step' = self.user.current_map.steps_for_climbing[0]
+        if ('randomsong' in x.step_type or 'speedlimit' in x.step_type) and self.user_play.song_grade < 5:
+            self.character_bonus_progress = -self.step_value / 2 / self.step_times
+            self.step_value = self.step_value / 2
+            self.user.current_map.reclimb(self.step_value)

latest version/database/database_initialize.py

@@ -1,605 +1,15 @@
-import sqlite3
-import time
-import json
+import os
+import sys
+from importlib import import_module
 
 # 数据库初始化文件，删掉arcaea_database.db文件后运行即可，谨慎使用
 
-ARCAEA_SERVER_VERSION = 'v2.10.0'
 
-
-def main(path='./'):
-    conn = sqlite3.connect(path+'arcaea_database.db')
-    c = conn.cursor()
-    c.execute('''create table if not exists config(id text primary key,
-    value text
-    );''')
-    c.execute('''insert into config values("version", :a);''',
-              {'a': ARCAEA_SERVER_VERSION})
-    c.execute('''create table if not exists user(user_id int primary key,
-    name text unique,
-    password text not null,
-    join_date char(20),
-    user_code char(10),
-    rating_ptt int,
-    character_id int,
-    is_skill_sealed int,
-    is_char_uncapped int,
-    is_char_uncapped_override int,
-    is_hide_rating int,
-    song_id text,
-    difficulty int,
-    score int,
-    shiny_perfect_count int,
-    perfect_count int,
-    near_count int,
-    miss_count int,
-    health int,
-    modifier int,
-    time_played int,
-    clear_type int,
-    rating real,
-    favorite_character int,
-    max_stamina_notification_enabled int,
-    current_map text,
-    ticket int,
-    prog_boost int,
-    email text,
-    world_rank_score int,
-    ban_flag text,
-    next_fragstam_ts int,
-    max_stamina_ts int,
-    stamina int,
-    world_mode_locked_end_ts int
-    );''')
-    c.execute('''create table if not exists login(access_token text,
-    user_id int,
-    login_time int,
-    login_ip text,
-    login_device text,
-    primary key(access_token, user_id)
-    );''')
-    c.execute('''create table if not exists friend(user_id_me int,
-    user_id_other int,
-    primary key (user_id_me, user_id_other)
-    );''')
-    c.execute('''create table if not exists best_score(user_id int,
-    song_id text,
-    difficulty int,
-    score int,
-    shiny_perfect_count int,
-    perfect_count int,
-    near_count int,
-    miss_count int,
-    health int,
-    modifier int,
-    time_played int,
-    best_clear_type int,
-    clear_type int,
-    rating real,
-    primary key(user_id, song_id, difficulty)
-    );''')
-    c.execute('''create table if not exists user_char(user_id int,
-    character_id int,
-    level int,
-    exp real,
-    is_uncapped int,
-    is_uncapped_override int,
-    primary key(user_id, character_id)
-    );''')
-    c.execute('''create table if not exists user_char_full(user_id int,
-    character_id int,
-    level int,
-    exp real,
-    is_uncapped int,
-    is_uncapped_override int,
-    primary key(user_id, character_id)
-    );''')
-    c.execute('''create table if not exists character(character_id int primary key,
-    name text,
-    max_level int,
-    frag1 real,
-    prog1 real,
-    overdrive1 real,
-    frag20 real,
-    prog20 real,
-    overdrive20 real,
-    frag30 real,
-    prog30 real,
-    overdrive30 real,
-    skill_id text,
-    skill_unlock_level int,
-    skill_requires_uncap int,
-    skill_id_uncap text,
-    char_type int,
-    is_uncapped int
-    );''')
-    c.execute('''create table if not exists char_item(character_id int,
-    item_id text,
-    type text,
-    amount int,
-    primary key(character_id, item_id, type)
-    );''')
-    c.execute('''create table if not exists recent30(user_id int primary key,
-    r0 real,
-    song_id0 text,
-    r1 real,
-    song_id1 text,
-    r2 real,
-    song_id2 text,
-    r3 real,
-    song_id3 text,
-    r4 real,
-    song_id4 text,
-    r5 real,
-    song_id5 text,
-    r6 real,
-    song_id6 text,
-    r7 real,
-    song_id7 text,
-    r8 real,
-    song_id8 text,
-    r9 real,
-    song_id9 text,
-    r10 real,
-    song_id10 text,
-    r11 real,
-    song_id11 text,
-    r12 real,
-    song_id12 text,
-    r13 real,
-    song_id13 text,
-    r14 real,
-    song_id14 text,
-    r15 real,
-    song_id15 text,
-    r16 real,
-    song_id16 text,
-    r17 real,
-    song_id17 text,
-    r18 real,
-    song_id18 text,
-    r19 real,
-    song_id19 text,
-    r20 real,
-    song_id20 text,
-    r21 real,
-    song_id21 text,
-    r22 real,
-    song_id22 text,
-    r23 real,
-    song_id23 text,
-    r24 real,
-    song_id24 text,
-    r25 real,
-    song_id25 text,
-    r26 real,
-    song_id26 text,
-    r27 real,
-    song_id27 text,
-    r28 real,
-    song_id28 text,
-    r29 real,
-    song_id29 text
-    );''')
-    c.execute('''create table if not exists user_world(user_id int,
-    map_id text,
-    curr_position int,
-    curr_capture real,
-    is_locked int,
-    primary key(user_id, map_id)
-    );''')
-    c.execute('''create table if not exists songplay_token(token text primary key,
-    user_id int,
-    song_id text,
-    difficulty int,
-    course_id text,
-    course_state int,
-    course_score int,
-    course_clear_type int,
-    stamina_multiply int,
-    fragment_multiply int,
-    prog_boost_multiply int
-    );''')
-    c.execute('''create table if not exists download_token(user_id int,
-    song_id text,
-    file_name text,
-    token text,
-    time int,
-    primary key(user_id, song_id, file_name)
-    );''')
-    c.execute('''create table if not exists user_download(user_id int,
-    time int,
-    token text,
-    primary key(user_id, time, token)
-    );''')
-    c.execute('''create table if not exists item(item_id text,
-    type text,
-    is_available int,
-    _id text,
-    primary key(item_id, type)
-    );''')
-    c.execute('''create table if not exists user_item(user_id int,
-    item_id text,
-    type text,
-    amount int,
-    primary key(user_id, item_id, type)
-    );''')
-    c.execute('''create table if not exists purchase(purchase_name text primary key,
-    price int,
-    orig_price int,
-    discount_from int,
-    discount_to int,
-    discount_reason text
-    );''')
-    c.execute('''create table if not exists purchase_item(purchase_name text,
-    item_id text,
-    type text,
-    amount int,
-    primary key(purchase_name, item_id, type)
-    );''')
-    c.execute('''create table if not exists user_save(user_id int primary key,
-    scores_data text,
-    clearlamps_data text,
-    clearedsongs_data text,
-    unlocklist_data text,
-    installid_data text,
-    devicemodelname_data text,
-    story_data text,
-    createdAt int,
-    finalestate_data text
-    );''')
-    c.execute('''create table if not exists present(present_id text primary key,
-    expire_ts int,
-    description text
-    );''')
-    c.execute('''create table if not exists user_present(user_id int,
-    present_id text,
-    primary key(user_id, present_id)
-    );''')
-    c.execute('''create table if not exists present_item(present_id text,
-    item_id text,
-    type text,
-    amount int,
-    primary key(present_id, item_id, type)
-    );''')
-    c.execute('''create table if not exists chart(song_id text primary key,
-    name text,
-    rating_pst int,
-    rating_prs int,
-    rating_ftr int,
-    rating_byn int
-    );''')
-    c.execute('''create table if not exists redeem(code text primary key,
-    type int
-    );''')
-    c.execute('''create table if not exists user_redeem(user_id int,
-    code text,
-    primary key(user_id, code)
-    );''')
-    c.execute('''create table if not exists redeem_item(code text,
-    item_id text,
-    type text,
-    amount int,
-    primary key(code, item_id, type)
-    );''')
-
-    c.execute('''create table if not exists role(role_id int primary key,
-    role_name text,
-    caption text
-    );''')
-    c.execute('''create table if not exists user_role(user_id int,
-    role_id int,
-    primary key(user_id, role_id)
-    );''')
-    c.execute('''create table if not exists power(power_id int primary key,
-    power_name text,
-    caption text
-    );''')
-    c.execute('''create table if not exists role_power(role_id int,
-    power_id int,
-    primary key(role_id, power_id)
-    );''')
-    c.execute('''create table if not exists api_login(user_id int,
-    token text,
-    login_time int,
-    login_ip text,
-    primary key(user_id, token)
-    );''')
-    c.execute('''create table if not exists course(course_id text primary key,
-    course_name text,
-    dan_name text,
-    style int,
-    gauge_requirement text,
-    flag_as_hidden_when_requirements_not_met int,
-    can_start int
-    );''')
-    c.execute('''create table if not exists user_course(user_id int,
-    course_id text,
-    high_score int,
-    best_clear_type int,
-    primary key(user_id, course_id)
-    );''')
-    c.execute('''create table if not exists course_chart(course_id text,
-    song_id text,
-    difficulty int,
-    flag_as_hidden int,
-    song_index int,
-    primary key(course_id, song_index)
-    );''')
-    c.execute('''create table if not exists course_requirement(course_id text,
-    required_id text,
-    primary key(course_id, required_id)
-    );''')
-    c.execute('''create table if not exists course_item(course_id text,
-    item_id text,
-    type text,
-    amount int,
-    primary key(course_id, item_id, type)
-    );''')
-
-    c.execute(
-        '''create index best_score_1 on best_score (song_id, difficulty);''')  # 排名查询优化
-    c.execute(
-        '''create index download_token_1 on download_token (song_id, file_name);''')  # 下载token判断优化
-
-    # 搭档初始化
-    char = ['hikari', 'tairitsu', 'kou', 'sapphire', 'lethe', 'hikari&tairitsu(reunion)', 'Tairitsu(Axium)', 'Tairitsu(Grievous Lady)', 'stella', 'Hikari & Fisica', 'ilith', 'eto', 'luna', 'shirabe', 'Hikari(Zero)', 'Hikari(Fracture)', 'Hikari(Summer)', 'Tairitsu(Summer)', 'Tairitsu & Trin',
-            'ayu', 'Eto & Luna', 'yume', 'Seine & Hikari', 'saya', 'Tairitsu & Chuni Penguin', 'Chuni Penguin', 'haruna', 'nono', 'MTA-XXX', 'MDA-21', 'kanae', 'Hikari(Fantasia)', 'Tairitsu(Sonata)', 'sia', 'DORO*C', 'Tairitsu(Tempest)', 'brillante', 'Ilith(Summer)', 'etude', 'Alice & Tenniel', 'Luna & Mia', 'areus', 'seele', 'isabelle', 'mir', 'lagrange', 'linka', 'nami', 'Saya & Elizabeth', 'lily', 'kanae(midsummer)', 'alice&tenniel(minuet)', 'tairitsu(elegy)', 'marija', 'vita', 'hikari(fatalis)', 'saki', 'setsuna']
-
-    skill_id = ['gauge_easy', '', '', '', 'note_mirror', 'skill_reunion', '', 'gauge_hard', 'frag_plus_10_pack_stellights', 'gauge_easy|frag_plus_15_pst&prs', 'gauge_hard|fail_frag_minus_100', 'frag_plus_5_side_light', 'visual_hide_hp', 'frag_plus_5_side_conflict', 'challenge_fullcombo_0gauge', 'gauge_overflow', 'gauge_easy|note_mirror', 'note_mirror', 'visual_tomato_pack_tonesphere',
-                'frag_rng_ayu', 'gaugestart_30|gaugegain_70', 'combo_100-frag_1', 'audio_gcemptyhit_pack_groovecoaster', 'gauge_saya', 'gauge_chuni', 'kantandeshou', 'gauge_haruna', 'frags_nono', 'gauge_pandora', 'gauge_regulus', 'omatsuri_daynight', '', '', 'sometimes(note_mirror|frag_plus_5)', 'scoreclear_aa|visual_scoregauge', 'gauge_tempest', 'gauge_hard', 'gauge_ilith_summer', '', 'note_mirror|visual_hide_far', 'frags_ongeki', 'gauge_areus', 'gauge_seele', 'gauge_isabelle', 'gauge_exhaustion', 'skill_lagrange', 'gauge_safe_10', 'frags_nami', 'skill_elizabeth', 'skill_lily', 'skill_kanae_midsummer', '', '', 'visual_ghost_skynotes', 'skill_vita', 'skill_fatalis', 'frags_ongeki_slash', 'frags_ongeki_hard']
-
-    skill_id_uncap = ['', '', 'frags_kou', '', 'visual_ink', '', '', '', '', '', '', 'eto_uncap', 'luna_uncap', 'shirabe_entry_fee',
-                      '', '', '', '', '', 'ayu_uncap', '', 'frags_yume', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
-
-    skill_unlock_level = [0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0,
-                          0, 0, 0, 8, 0, 14, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 8, 0]
-
-    frag1 = [55, 55, 60, 50, 47, 79, 47, 57, 41, 22, 50, 54, 60, 56, 78, 42, 41, 61, 52, 50, 52, 32,
-             42, 55, 45, 58, 43, 0.5, 68, 50, 62, 45, 45, 52, 44, 27, 59, 0, 45, 50, 50, 47, 47, 61, 43, 42, 38, 25, 58, 50, 61, 45, 45, 38, 34, 27, 18, 56]
-
-    prog1 = [35, 55, 47, 50, 60, 70, 60, 70, 58, 45, 70, 45, 42, 46, 61, 67, 49, 44, 28, 45, 24, 46, 52,
-             59, 62, 33, 58, 25, 63, 69, 50, 45, 45, 51, 34, 70, 62, 70, 45, 32, 32, 61, 47, 47, 37, 42, 50, 50, 45, 41, 61, 45, 45, 58, 50, 130, 18, 57]
-
-    overdrive1 = [35, 55, 25, 50, 47, 70, 72, 57, 41, 7, 10, 32, 65, 31, 61, 53, 31, 47, 38, 12, 39, 18,
-                  48, 65, 45, 55, 44, 25, 46, 44, 33, 45, 45, 37, 25, 27, 50, 20, 45, 63, 21, 47, 61, 47, 65, 80, 38, 30, 49, 15, 34, 45, 45, 38, 67, 120, 44, 33]
-
-    frag20 = [78, 80, 90, 75, 70, 79, 70, 79, 65, 40, 50, 80, 90, 82, 0, 61, 67, 92, 85, 50, 86, 52,
-              65, 85, 67, 88, 64, 0.5, 95, 70, 95, 50, 80, 87, 71, 50, 85, 0, 80, 75, 50, 70, 70, 90, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 34, 85]
-
-    prog20 = [61, 80, 70, 75, 90, 70, 90, 102, 84, 78, 105, 67, 63, 68, 0, 99, 80, 66, 46, 83, 40, 73,
-              80, 90, 93, 50, 86, 78, 89, 98, 75, 80, 50, 64, 55, 100, 90, 110, 80, 50, 74, 90, 70, 70, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 34, 86]
-
-    overdrive20 = [61, 80, 47, 75, 70, 70, 95, 79, 65, 31, 50, 59, 90, 58, 0, 78, 50, 70, 62, 49, 64,
-                   46, 73, 95, 67, 84, 70, 78, 69, 70, 50, 80, 80, 63, 25, 50, 72, 55, 50, 95, 55, 70, 90, 70, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 84, 50]
-
-    frag30 = [88, 90, 100, 75, 80, 89, 70, 79, 65, 40, 50, 90, 100, 92, 0, 61, 67, 92, 85, 50, 86, 62,
-              65, 85, 67, 88, 74, 0.5, 105, 80, 95, 50, 80, 87, 71, 50, 95, 0, 80, 75, 50, 70, 80, 100, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 34, 85]
-
-    prog30 = [71, 90, 80, 75, 100, 80, 90, 102, 84, 78, 105, 77, 73, 78, 0, 99, 80, 66, 46, 93, 40, 83,
-              80, 90, 93, 50, 96, 88, 99, 108, 75, 80, 50, 64, 55, 100, 100, 110, 80, 50, 74, 90, 80, 80, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 34, 86]
-
-    overdrive30 = [71, 90, 57, 75, 80, 80, 95, 79, 65, 31, 50, 69, 100, 68, 0, 78, 50, 70, 62, 59, 64,
-                   56, 73, 95, 67, 84, 80, 88, 79, 80, 50, 80, 80, 63, 25, 50, 82, 55, 50, 95, 55, 70, 100, 80, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 84, 50]
-
-    char_type = [1, 0, 0, 0, 0, 0, 0, 2, 0, 1, 2, 0, 0, 0, 2, 3, 1, 0, 0, 0, 1,
-                 0, 0, 0, 0, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 2, 2, 2, 0, 0, 0, 2, 2, 2, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 2, 3, 0, 2]
-
-    char_core = {
-        0: [{'core_id': 'core_hollow', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
-        1: [{'core_id': 'core_hollow', 'amount': 5}, {'core_id': 'core_desolate', 'amount': 25}],
-        2: [{'core_id': 'core_hollow', 'amount': 5}, {'core_id': 'core_crimson', 'amount': 25}],
-        4: [{'core_id': 'core_ambivalent', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
-        13: [{'core_id': 'core_scarlet', 'amount': 30}],
-        21: [{'core_id': 'core_scarlet', 'amount': 30}],
-        26: [{'core_id': 'core_chunithm', 'amount': 15}],
-        27: [{'core_id': 'core_chunithm', 'amount': 15}],
-        28: [{'core_id': 'core_chunithm', 'amount': 15}],
-        29: [{'core_id': 'core_chunithm', 'amount': 15}],
-        36: [{'core_id': 'core_chunithm', 'amount': 15}],
-        42: [{'core_id': 'core_chunithm', 'amount': 15}],
-        43: [{'core_id': 'core_chunithm', 'amount': 15}],
-        11: [{'core_id': 'core_binary', 'amount': 25}, {'core_id': 'core_hollow', 'amount': 5}],
-        12: [{'core_id': 'core_binary', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
-        19: [{'core_id': 'core_colorful', 'amount': 30}]
-    }
-
-    for i in range(0, 58):
-        skill_requires_uncap = 1 if i == 2 else 0
-
-        if i in [0, 1, 2, 4, 13, 26, 27, 28, 29, 36, 21, 42, 43, 11, 12, 19, 5]:
-            sql = '''insert into character values(?,?,30,?,?,?,?,?,?,?,?,?,?,?,?,?,?,1)'''
-            c.execute(sql, (i, char[i], frag1[i], prog1[i], overdrive1[i], frag20[i], prog20[i], overdrive20[i],
-                            frag30[i], prog30[i], overdrive30[i], skill_id[i], skill_unlock_level[i], skill_requires_uncap, skill_id_uncap[i], char_type[i]))
-        else:
-            sql = '''insert into character values(?,?,20,?,?,?,?,?,?,?,?,?,?,?,?,?,?,0)'''
-            c.execute(sql, (i, char[i], frag1[i], prog1[i], overdrive1[i], frag20[i], prog20[i], overdrive20[i],
-                            frag30[i], prog30[i], overdrive30[i], skill_id[i], skill_unlock_level[i], skill_requires_uncap, skill_id_uncap[i], char_type[i]))
-
-    c.execute('''insert into character values(?,?,20,?,?,?,?,?,?,?,?,?,?,?,?,?,?,0)''', (99,
-                                                                                         'shirahime', 38, 33, 28, 66, 58, 50, 66, 58, 50, 'frags_preferred_song', 0, 0, '', 0))
-
-    for i in char_core:
-        for j in char_core[i]:
-            c.execute('''insert into char_item values(?,?,'core',?)''',
-                      (i, j['core_id'], j['amount']))
-    cores = ['core_hollow', 'core_desolate', 'core_chunithm', 'core_crimson',
-             'core_ambivalent', 'core_scarlet', 'core_groove', 'core_generic', 'core_binary', 'core_colorful', 'core_course_skip_purchase']
-
-    for i in cores:
-        c.execute('''insert into item values(?,"core",1,'')''', (i,))
-
-    world_songs = ["babaroque", "shadesoflight", "kanagawa", "lucifer", "anokumene", "ignotus", "rabbitintheblackroom", "qualia", "redandblue", "bookmaker", "darakunosono", "espebranch", "blacklotus", "givemeanightmare", "vividtheory", "onefr", "gekka", "vexaria3", "infinityheaven3", "fairytale3", "goodtek3", "suomi", "rugie", "faintlight", "harutopia", "goodtek", "dreaminattraction", "syro", "diode", "freefall", "grimheart", "blaster",
-                   "cyberneciacatharsis", "monochromeprincess", "revixy", "vector", "supernova", "nhelv", "purgatorium3", "dement3", "crossover", "guardina", "axiumcrisis", "worldvanquisher", "sheriruth", "pragmatism", "gloryroad", "etherstrike", "corpssansorganes", "lostdesire", "blrink", "essenceoftwilight", "lapis", "solitarydream", "lumia3", "purpleverse", "moonheart3", "glow", "enchantedlove", "take", "lifeispiano", "vandalism", "nexttoyou3", "lostcivilization3", "turbocharger", "bookmaker3", "laqryma3", "kyogenkigo", "hivemind", "seclusion", "quonwacca3", "bluecomet", "energysynergymatrix", "gengaozo", "lastendconductor3", "antithese3", "qualia3", "kanagawa3", "heavensdoor3", "pragmatism3", "nulctrl", "avril", "ddd", "merlin3", "omakeno3", "nekonote", "sanskia", 'altair', 'mukishitsu', 'trapcrow', 'redandblue3', 'ignotus3', 'singularity3', 'dropdead3', 'arcahv']
-    for i in world_songs:
-        c.execute('''insert into item values(?,"world_song",1,'')''', (i,))
-
-    world_unlocks = ["scenery_chap1", "scenery_chap2",
-                     "scenery_chap3", "scenery_chap4", "scenery_chap5", "scenery_chap6", "scenery_chap7"]
-    for i in world_unlocks:
-        c.execute('''insert into item values(?,"world_unlock",1,'')''', (i,))
-
-    course_banners = ['course_banner_' + str(i) for i in range(1, 12)]
-    for i in course_banners:
-        c.execute('''insert into item values(?,"course_banner",1,'')''', (i,))
-
-    c.execute('''insert into item values(?,?,?,?)''',
-              ('fragment', 'fragment', 1, ''))
-    c.execute('''insert into item values(?,?,?,?)''',
-              ('memory', 'memory', 1, ''))
-    c.execute('''insert into item values(?,?,?,?)''',
-              ('anni5tix', 'anni5tix', 1, ''))
-
-    def insert_items(c, items):
-        # 物品数据导入
-        for i in items:
-            if 'discount_from' not in i:
-                discount_from = -1
-            else:
-                discount_from = i['discount_from']
-            if 'discount_to' not in i:
-                discount_to = -1
-            else:
-                discount_to = i['discount_to']
-            if 'discount_reason' not in i:
-                discount_reason = ''
-            else:
-                discount_reason = i['discount_reason']
-            c.execute('''insert into purchase values(?,?,?,?,?,?)''',
-                      (i['name'], i['price'], i['orig_price'], discount_from, discount_to, discount_reason))
-            for j in i['items']:
-                if "_id" not in j:
-                    _id = ''
-                else:
-                    _id = j['_id']
-                c.execute(
-                    '''select exists(select * from item where item_id=?)''', (j['id'],))
-                if c.fetchone() == (0,):
-                    c.execute('''insert into item values(?,?,?,?)''',
-                              (j['id'], j['type'], j['is_available'], _id))
-                if 'amount' in j:
-                    amount = j['amount']
-                else:
-                    amount = 1
-                c.execute('''insert into purchase_item values(?,?,?,?)''',
-                          (i['name'], j['id'], j['type'], amount))
-
-    # item初始化
-    try:
-        f = open(path+'singles.json', 'r')
-        singles = json.load(f)
-        f.close()
-        insert_items(c, singles)
-    except:
-        pass
-
-    try:
-        f = open(path+'packs.json', 'r')
-        packs = json.load(f)
-        f.close()
-        insert_items(c, packs)
-        c.execute('''select exists(select * from item where item_id='epilogue')''')
-        if c.fetchone() == (0,):
-            c.execute('''insert into item values('epilogue','pack',1,'')''')
-    except:
-        pass
-
-    # course初始化
-    try:
-        f = open(path+'courses.json', 'r', encoding="utf-8")
-        courses = json.load(f)
-        f.close()
-    except:
-        courses = []
-
-    if courses:
-        if path == './':
-            # 有什么好办法吗
-            import sys
-            sys.path.append('..')
-        from core.course import Course
-        for i in courses:
-            x = Course(c).from_dict(i)
-            x.insert_all()
-
-        # api权限与权限组初始化
-    role = ['admin', 'user', 'selecter']
-    role_caption = ['管理员', '用户', '查询接口']
-
-    power = ['select', 'select_me', 'change', 'change_me', 'grant',
-             'grant_inf', 'select_song_rank', 'select_song_info']
-    power_caption = ['总体查询权限', '自我查询权限', '总体修改权限',
-                     '自我修改权限', '授权权限', '下级授权权限', '歌曲排行榜查询权限', '歌曲信息查询权限']
-
-    role_power = {'0': [0, 1, 3, 5, 6, 7],
-                  '1': [1, 3, 6, 7],
-                  '2': [0]
-                  }
-
-    for i in range(0, len(role)):
-        c.execute('''insert into role values(:a,:b,:c)''', {
-            'a': i, 'b': role[i], 'c': role_caption[i]})
-    for i in range(0, len(power)):
-        c.execute('''insert into power values(:a,:b,:c)''', {
-            'a': i, 'b': power[i], 'c': power_caption[i]})
-    for i in role_power:
-        for j in role_power[i]:
-            c.execute('''insert into role_power values(:a,:b)''',
-                      {'a': int(i), 'b': j})
-
-    conn.commit()
-    conn.close()
-
-    def arc_register(name: str):
-        def build_user_code(c):
-            return '123456789'
-
-        def build_user_id(c):
-            return 2000000
-
-        def insert_user_char(c, user_id):
-            # 为用户添加可用角色
-            c.execute('''insert into user_char values(?,?,?,?,?,?)''',
-                      (user_id, 0, 30, 25000, 1, 0))
-            c.execute('''insert into user_char values(?,?,?,?,?,?)''',
-                      (user_id, 1, 30, 25000, 1, 0))
-
-            c.execute(
-                '''select character_id, max_level, is_uncapped from character''')
-            x = c.fetchall()
-            if x:
-                for i in x:
-                    exp = 25000 if i[1] == 30 else 10000
-                    c.execute('''insert into user_char_full values(?,?,?,?,?,?)''',
-                              (user_id, i[0], i[1], exp, i[2], 0))
-
-        conn = sqlite3.connect(path+'arcaea_database.db')
-        c = conn.cursor()
-        hash_pwd = '41e5653fc7aeb894026d6bb7b2db7f65902b454945fa8fd65a6327047b5277fb'
-        c.execute(
-            '''select exists(select * from user where name = :name)''', {'name': name})
-        if c.fetchone() == (0,):
-            user_code = build_user_code(c)
-            user_id = build_user_id(c)
-            now = int(time.time() * 1000)
-            c.execute('''insert into user(user_id, name, password, join_date, user_code, rating_ptt,
-            character_id, is_skill_sealed, is_char_uncapped, is_char_uncapped_override, is_hide_rating, favorite_character, max_stamina_notification_enabled, current_map, ticket)
-            values(:user_id, :name, :password, :join_date, :user_code, 1250, 1, 0, 1, 0, 0, -1, 0, '', 114514)
-            ''', {'user_code': user_code, 'user_id': user_id, 'join_date': now, 'name': name, 'password': hash_pwd})
-            c.execute('''insert into recent30(user_id) values(:user_id)''', {
-                'user_id': user_id})
-            c.execute(
-                '''insert into best_score values(2000000,'vexaria',3,10000000,100,0,0,0,100,0,1599667200,3,3,10.8)''')
-            insert_user_char(c, user_id)
-            conn.commit()
-            conn.close()
-            return None
-        else:
-            conn.commit()
-            conn.close()
-            return None
-
-    arc_register('admin')
+def main(core_path: str = '../', db_path: str = './arcaea_database.db', init_folder_path: str = './init'):
+    sys.path.append(os.path.join(sys.path[0], core_path))
+    sys.path.append(os.path.join(sys.path[0], core_path, './core/'))
+    init = import_module('init').DatabaseInit(db_path, init_folder_path)
+    init.init()
 
 
 if __name__ == '__main__':

latest version/database/init/arc_data.py

@@ -0,0 +1,86 @@
+class InitData:
+    char = ['hikari', 'tairitsu', 'kou', 'sapphire', 'lethe', 'hikari&tairitsu(reunion)', 'Tairitsu(Axium)', 'Tairitsu(Grievous Lady)', 'stella', 'Hikari & Fisica', 'ilith', 'eto', 'luna', 'shirabe', 'Hikari(Zero)', 'Hikari(Fracture)', 'Hikari(Summer)', 'Tairitsu(Summer)', 'Tairitsu & Trin',
+            'ayu', 'Eto & Luna', 'yume', 'Seine & Hikari', 'saya', 'Tairitsu & Chuni Penguin', 'Chuni Penguin', 'haruna', 'nono', 'MTA-XXX', 'MDA-21', 'kanae', 'Hikari(Fantasia)', 'Tairitsu(Sonata)', 'sia', 'DORO*C', 'Tairitsu(Tempest)', 'brillante', 'Ilith(Summer)', 'etude', 'Alice & Tenniel', 'Luna & Mia', 'areus', 'seele', 'isabelle', 'mir', 'lagrange', 'linka', 'nami', 'Saya & Elizabeth', 'lily', 'kanae(midsummer)', 'alice&tenniel(minuet)', 'tairitsu(elegy)', 'marija', 'vita', 'hikari(fatalis)', 'saki', 'setsuna', 'amane', 'kou(winter)']
+
+    skill_id = ['gauge_easy', '', '', '', 'note_mirror', 'skill_reunion', '', 'gauge_hard', 'frag_plus_10_pack_stellights', 'gauge_easy|frag_plus_15_pst&prs', 'gauge_hard|fail_frag_minus_100', 'frag_plus_5_side_light', 'visual_hide_hp', 'frag_plus_5_side_conflict', 'challenge_fullcombo_0gauge', 'gauge_overflow', 'gauge_easy|note_mirror', 'note_mirror', 'visual_tomato_pack_tonesphere',
+                'frag_rng_ayu', 'gaugestart_30|gaugegain_70', 'combo_100-frag_1', 'audio_gcemptyhit_pack_groovecoaster', 'gauge_saya', 'gauge_chuni', 'kantandeshou', 'gauge_haruna', 'frags_nono', 'gauge_pandora', 'gauge_regulus', 'omatsuri_daynight', '', '', 'sometimes(note_mirror|frag_plus_5)', 'scoreclear_aa|visual_scoregauge', 'gauge_tempest', 'gauge_hard', 'gauge_ilith_summer', '', 'note_mirror|visual_hide_far', 'frags_ongeki', 'gauge_areus', 'gauge_seele', 'gauge_isabelle', 'gauge_exhaustion', 'skill_lagrange', 'gauge_safe_10', 'frags_nami', 'skill_elizabeth', 'skill_lily', 'skill_kanae_midsummer', '', '', 'visual_ghost_skynotes', 'skill_vita', 'skill_fatalis', 'frags_ongeki_slash', 'frags_ongeki_hard', 'skill_amane', 'skill_kou_winter']
+
+    skill_id_uncap = ['', '', 'frags_kou', '', 'visual_ink', '', '', '', '', '', '', 'eto_uncap', 'luna_uncap', 'shirabe_entry_fee',
+                      '', '', '', '', '', 'ayu_uncap', '', 'frags_yume', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
+
+    skill_unlock_level = [0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0,
+                          0, 0, 0, 8, 0, 14, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 8, 0, 0, 0]
+
+    frag1 = [55, 55, 60, 50, 47, 79, 47, 57, 41, 22, 50, 54, 60, 56, 78, 42, 41, 61, 52, 50, 52, 32,
+             42, 55, 45, 58, 43, 0.5, 68, 50, 62, 45, 45, 52, 44, 27, 59, 0, 45, 50, 50, 47, 47, 61, 43, 42, 38, 25, 58, 50, 61, 45, 45, 38, 34, 27, 18, 56, 47, 30]
+
+    prog1 = [35, 55, 47, 50, 60, 70, 60, 70, 58, 45, 70, 45, 42, 46, 61, 67, 49, 44, 28, 45, 24, 46, 52,
+             59, 62, 33, 58, 25, 63, 69, 50, 45, 45, 51, 34, 70, 62, 70, 45, 32, 32, 61, 47, 47, 37, 42, 50, 50, 45, 41, 61, 45, 45, 58, 50, 130, 18, 57, 55, 50]
+
+    overdrive1 = [35, 55, 25, 50, 47, 70, 72, 57, 41, 7, 10, 32, 65, 31, 61, 53, 31, 47, 38, 12, 39, 18,
+                  48, 65, 45, 55, 44, 25, 46, 44, 33, 45, 45, 37, 25, 27, 50, 20, 45, 63, 21, 47, 61, 47, 65, 80, 38, 30, 49, 15, 34, 45, 45, 38, 67, 120, 44, 33, 55, 50]
+
+    frag20 = [78, 80, 90, 75, 70, 79, 70, 79, 65, 40, 50, 80, 90, 82, 0, 61, 67, 92, 85, 50, 86, 52,
+              65, 85, 67, 88, 64, 0.5, 95, 70, 95, 50, 80, 87, 71, 50, 85, 0, 80, 75, 50, 70, 70, 90, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 35, 85, 47, 50]
+
+    prog20 = [61, 80, 70, 75, 90, 70, 90, 102, 84, 78, 105, 67, 63, 68, 0, 99, 80, 66, 46, 83, 40, 73,
+              80, 90, 93, 50, 86, 78, 89, 98, 75, 80, 50, 64, 55, 100, 90, 110, 80, 50, 74, 90, 70, 70, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 35, 86, 92, 80]
+
+    overdrive20 = [61, 80, 47, 75, 70, 70, 95, 79, 65, 31, 50, 59, 90, 58, 0, 78, 50, 70, 62, 49, 64,
+                   46, 73, 95, 67, 84, 70, 78, 69, 70, 50, 80, 80, 63, 25, 50, 72, 55, 50, 95, 55, 70, 90, 70, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 85, 50, 92, 50]
+
+    frag30 = [88, 90, 100, 75, 80, 89, 70, 79, 65, 40, 50, 90, 100, 92, 0, 61, 67, 92, 85, 50, 86, 62,
+              65, 85, 67, 88, 74, 0.5, 105, 80, 95, 50, 80, 87, 71, 50, 95, 0, 80, 75, 50, 70, 80, 100, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 35, 85, 47, 50]
+
+    prog30 = [71, 90, 80, 75, 100, 80, 90, 102, 84, 78, 105, 77, 73, 78, 0, 99, 80, 66, 46, 93, 40, 83,
+              80, 90, 93, 50, 96, 88, 99, 108, 75, 80, 50, 64, 55, 100, 100, 110, 80, 50, 74, 90, 80, 80, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 35, 86, 92, 80]
+
+    overdrive30 = [71, 90, 57, 75, 80, 80, 95, 79, 65, 31, 50, 69, 100, 68, 0, 78, 50, 70, 62, 59, 64,
+                   56, 73, 95, 67, 84, 80, 88, 79, 80, 50, 80, 80, 63, 25, 50, 82, 55, 50, 95, 55, 70, 100, 80, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 85, 50, 92, 50]
+
+    char_type = [1, 0, 0, 0, 0, 0, 0, 2, 0, 1, 2, 0, 0, 0, 2, 3, 1, 0, 0, 0, 1,
+                 0, 0, 0, 0, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 2, 2, 2, 0, 0, 0, 2, 2, 2, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 2, 3, 0, 2, 2, 0]
+
+    char_core = {
+        0: [{'core_id': 'core_hollow', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
+        1: [{'core_id': 'core_hollow', 'amount': 5}, {'core_id': 'core_desolate', 'amount': 25}],
+        2: [{'core_id': 'core_hollow', 'amount': 5}, {'core_id': 'core_crimson', 'amount': 25}],
+        4: [{'core_id': 'core_ambivalent', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
+        13: [{'core_id': 'core_scarlet', 'amount': 30}],
+        21: [{'core_id': 'core_scarlet', 'amount': 30}],
+        26: [{'core_id': 'core_chunithm', 'amount': 15}],
+        27: [{'core_id': 'core_chunithm', 'amount': 15}],
+        28: [{'core_id': 'core_chunithm', 'amount': 15}],
+        29: [{'core_id': 'core_chunithm', 'amount': 15}],
+        36: [{'core_id': 'core_chunithm', 'amount': 15}],
+        42: [{'core_id': 'core_chunithm', 'amount': 15}],
+        43: [{'core_id': 'core_chunithm', 'amount': 15}],
+        11: [{'core_id': 'core_binary', 'amount': 25}, {'core_id': 'core_hollow', 'amount': 5}],
+        12: [{'core_id': 'core_binary', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
+        19: [{'core_id': 'core_colorful', 'amount': 30}]
+    }
+
+    cores = ['core_hollow', 'core_desolate', 'core_chunithm', 'core_crimson',
+             'core_ambivalent', 'core_scarlet', 'core_groove', 'core_generic', 'core_binary', 'core_colorful', 'core_course_skip_purchase']
+
+    world_songs = ["babaroque", "shadesoflight", "kanagawa", "lucifer", "anokumene", "ignotus", "rabbitintheblackroom", "qualia", "redandblue", "bookmaker", "darakunosono", "espebranch", "blacklotus", "givemeanightmare", "vividtheory", "onefr", "gekka", "vexaria3", "infinityheaven3", "fairytale3", "goodtek3", "suomi", "rugie", "faintlight", "harutopia", "goodtek", "dreaminattraction", "syro", "diode", "freefall", "grimheart", "blaster",
+                   "cyberneciacatharsis", "monochromeprincess", "revixy", "vector", "supernova", "nhelv", "purgatorium3", "dement3", "crossover", "guardina", "axiumcrisis", "worldvanquisher", "sheriruth", "pragmatism", "gloryroad", "etherstrike", "corpssansorganes", "lostdesire", "blrink", "essenceoftwilight", "lapis", "solitarydream", "lumia3", "purpleverse", "moonheart3", "glow", "enchantedlove", "take", "lifeispiano", "vandalism", "nexttoyou3", "lostcivilization3", "turbocharger", "bookmaker3", "laqryma3", "kyogenkigo", "hivemind", "seclusion", "quonwacca3", "bluecomet", "energysynergymatrix", "gengaozo", "lastendconductor3", "antithese3", "qualia3", "kanagawa3", "heavensdoor3", "pragmatism3", "nulctrl", "avril", "ddd", "merlin3", "omakeno3", "nekonote", "sanskia", 'altair', 'mukishitsu', 'trapcrow', 'redandblue3', 'ignotus3', 'singularity3', 'dropdead3', 'arcahv', 'freefall3', 'partyvinyl3']
+
+    world_unlocks = ["scenery_chap1", "scenery_chap2",
+                     "scenery_chap3", "scenery_chap4", "scenery_chap5", "scenery_chap6", "scenery_chap7"]
+
+    course_banners = ['course_banner_' + str(i) for i in range(1, 12)]
+
+    role = ['system', 'admin', 'user', 'selecter']
+    role_caption = ['系统', '管理员', '用户', '查询接口']
+
+    power = ['system', 'select', 'select_me', 'change', 'change_me', 'grant',
+             'grant_inf', 'select_song_rank', 'select_song_info', 'select_song_rank_top']
+    power_caption = ['系统权限', '总体查询权限', '自我查询权限', '总体修改权限',
+                     '自我修改权限', '授权权限', '下级授权权限', '歌曲排行榜查询权限', '歌曲信息查询权限', '歌曲排行榜有限查询权限']
+
+    role_power = {'system': power,
+                  'admin': ['select', 'select_me', 'change_me', 'grant_inf', 'select_song_rank_top', 'select_song_info'],
+                  'user': ['select_me', 'change_me', 'select_song_rank', 'select_song_info'],
+                  'selector': ['select']
+                  }

latest version/database/init/log_tables.sql

@@ -0,0 +1,25 @@
+create table if not exists cache(key text primary key,
+value text,
+expire_time int
+);
+create table if not exists user_score(user_id int,
+song_id text,
+difficulty int,
+time_played int,
+score int,
+shiny_perfect_count int,
+perfect_count int,
+near_count int,
+miss_count int,
+health int,
+modifier int,
+clear_type int,
+rating real,
+primary key(user_id, song_id, difficulty, time_played)
+);
+
+create index if not exists user_score_1 on user_score (song_id, difficulty);
+create index if not exists user_score_2 on user_score (time_played);
+
+PRAGMA journal_mode = WAL;
+PRAGMA default_cache_size = 4000;

latest version/database/init/singles.json

@@ -1216,5 +1216,59 @@
 		],
 		"orig_price": 100,
 		"price": 100
+	},
+	{
+		"name": "freemyself",
+		"items": [
+			{
+				"type": "single",
+				"id": "freemyself",
+				"is_available": true
+			},
+			{
+				"type": "core",
+				"amount": 1,
+				"id": "core_generic",
+				"is_available": true
+			}
+		],
+		"orig_price": 100,
+		"price": 100
+	},
+	{
+		"name": "cocorocosmetic",
+		"items": [
+			{
+				"type": "single",
+				"id": "cocorocosmetic",
+				"is_available": true
+			},
+			{
+				"type": "core",
+				"amount": 1,
+				"id": "core_generic",
+				"is_available": true
+			}
+		],
+		"orig_price": 100,
+		"price": 100
+	},
+	{
+		"name": "capella",
+		"items": [
+			{
+				"type": "single",
+				"id": "capella",
+				"is_available": true
+			},
+			{
+				"type": "core",
+				"amount": 1,
+				"id": "core_generic",
+				"is_available": true
+			}
+		],
+		"orig_price": 100,
+		"price": 100
 	}
 ]

latest version/database/init/tables.sql

@@ -0,0 +1,312 @@
+create table if not exists config(id text primary key, value text);
+create table if not exists user(user_id int primary key,
+name text unique,
+password text,
+join_date char(20),
+user_code char(10),
+rating_ptt int,
+character_id int,
+is_skill_sealed int,
+is_char_uncapped int,
+is_char_uncapped_override int,
+is_hide_rating int,
+song_id text,
+difficulty int,
+score int,
+shiny_perfect_count int,
+perfect_count int,
+near_count int,
+miss_count int,
+health int,
+modifier int,
+time_played int,
+clear_type int,
+rating real,
+favorite_character int,
+max_stamina_notification_enabled int,
+current_map text,
+ticket int,
+prog_boost int,
+email text,
+world_rank_score int,
+ban_flag text,
+next_fragstam_ts int,
+max_stamina_ts int,
+stamina int,
+world_mode_locked_end_ts int
+);
+create table if not exists login(access_token text,
+user_id int,
+login_time int,
+login_ip text,
+login_device text,
+primary key(access_token, user_id)
+);
+create table if not exists friend(user_id_me int,
+user_id_other int,
+primary key (user_id_me, user_id_other)
+);
+create table if not exists best_score(user_id int,
+song_id text,
+difficulty int,
+score int,
+shiny_perfect_count int,
+perfect_count int,
+near_count int,
+miss_count int,
+health int,
+modifier int,
+time_played int,
+best_clear_type int,
+clear_type int,
+rating real,
+primary key(user_id, song_id, difficulty)
+);
+create table if not exists user_char(user_id int,
+character_id int,
+level int,
+exp real,
+is_uncapped int,
+is_uncapped_override int,
+primary key(user_id, character_id)
+);
+create table if not exists user_char_full(user_id int,
+character_id int,
+level int,
+exp real,
+is_uncapped int,
+is_uncapped_override int,
+primary key(user_id, character_id)
+);
+create table if not exists character(character_id int primary key,
+name text,
+max_level int,
+frag1 real,
+prog1 real,
+overdrive1 real,
+frag20 real,
+prog20 real,
+overdrive20 real,
+frag30 real,
+prog30 real,
+overdrive30 real,
+skill_id text,
+skill_unlock_level int,
+skill_requires_uncap int,
+skill_id_uncap text,
+char_type int,
+is_uncapped int
+);
+create table if not exists char_item(character_id int,
+item_id text,
+type text,
+amount int,
+primary key(character_id, item_id, type)
+);
+create table if not exists recent30(user_id int primary key,
+r0 real,
+song_id0 text,
+r1 real,
+song_id1 text,
+r2 real,
+song_id2 text,
+r3 real,
+song_id3 text,
+r4 real,
+song_id4 text,
+r5 real,
+song_id5 text,
+r6 real,
+song_id6 text,
+r7 real,
+song_id7 text,
+r8 real,
+song_id8 text,
+r9 real,
+song_id9 text,
+r10 real,
+song_id10 text,
+r11 real,
+song_id11 text,
+r12 real,
+song_id12 text,
+r13 real,
+song_id13 text,
+r14 real,
+song_id14 text,
+r15 real,
+song_id15 text,
+r16 real,
+song_id16 text,
+r17 real,
+song_id17 text,
+r18 real,
+song_id18 text,
+r19 real,
+song_id19 text,
+r20 real,
+song_id20 text,
+r21 real,
+song_id21 text,
+r22 real,
+song_id22 text,
+r23 real,
+song_id23 text,
+r24 real,
+song_id24 text,
+r25 real,
+song_id25 text,
+r26 real,
+song_id26 text,
+r27 real,
+song_id27 text,
+r28 real,
+song_id28 text,
+r29 real,
+song_id29 text
+);
+create table if not exists user_world(user_id int,
+map_id text,
+curr_position int,
+curr_capture real,
+is_locked int,
+primary key(user_id, map_id)
+);
+create table if not exists songplay_token(token text primary key,
+user_id int,
+song_id text,
+difficulty int,
+course_id text,
+course_state int,
+course_score int,
+course_clear_type int,
+stamina_multiply int,
+fragment_multiply int,
+prog_boost_multiply int
+);
+create table if not exists item(item_id text,
+type text,
+is_available int,
+primary key(item_id, type)
+);
+create table if not exists user_item(user_id int,
+item_id text,
+type text,
+amount int,
+primary key(user_id, item_id, type)
+);
+create table if not exists purchase(purchase_name text primary key,
+price int,
+orig_price int,
+discount_from int,
+discount_to int,
+discount_reason text
+);
+create table if not exists purchase_item(purchase_name text,
+item_id text,
+type text,
+amount int,
+primary key(purchase_name, item_id, type)
+);
+create table if not exists user_save(user_id int primary key,
+scores_data text,
+clearlamps_data text,
+clearedsongs_data text,
+unlocklist_data text,
+installid_data text,
+devicemodelname_data text,
+story_data text,
+createdAt int,
+finalestate_data text
+);
+create table if not exists present(present_id text primary key,
+expire_ts int,
+description text
+);
+create table if not exists user_present(user_id int,
+present_id text,
+primary key(user_id, present_id)
+);
+create table if not exists present_item(present_id text,
+item_id text,
+type text,
+amount int,
+primary key(present_id, item_id, type)
+);
+create table if not exists chart(song_id text primary key,
+name text,
+rating_pst int,
+rating_prs int,
+rating_ftr int,
+rating_byn int
+);
+create table if not exists redeem(code text primary key,
+type int
+);
+create table if not exists user_redeem(user_id int,
+code text,
+primary key(user_id, code)
+);
+create table if not exists redeem_item(code text,
+item_id text,
+type text,
+amount int,
+primary key(code, item_id, type)
+);
+
+create table if not exists role(role_id text primary key,
+caption text
+);
+create table if not exists user_role(user_id int,
+role_id text,
+primary key(user_id, role_id)
+);
+create table if not exists power(power_id text primary key,
+caption text
+);
+create table if not exists role_power(role_id text,
+power_id text,
+primary key(role_id, power_id)
+);
+create table if not exists api_login(user_id int,
+token text,
+login_time int,
+login_ip text,
+primary key(user_id, token)
+);
+create table if not exists course(course_id text primary key,
+course_name text,
+dan_name text,
+style int,
+gauge_requirement text,
+flag_as_hidden_when_requirements_not_met int,
+can_start int
+);
+create table if not exists user_course(user_id int,
+course_id text,
+high_score int,
+best_clear_type int,
+primary key(user_id, course_id)
+);
+create table if not exists course_chart(course_id text,
+song_id text,
+difficulty int,
+flag_as_hidden int,
+song_index int,
+primary key(course_id, song_index)
+);
+create table if not exists course_requirement(course_id text,
+required_id text,
+primary key(course_id, required_id)
+);
+create table if not exists course_item(course_id text,
+item_id text,
+type text,
+amount int,
+primary key(course_id, item_id, type)
+);
+
+create index if not exists best_score_1 on best_score (song_id, difficulty);
+
+PRAGMA journal_mode = WAL;
+PRAGMA default_cache_size = 8000;

latest version/main.py

@@ -1,6 +1,20 @@
 # encoding: utf-8
 
 import os
+from importlib import import_module
+
+from core.config_manager import Config, ConfigManager
+
+if os.path.exists('config.py') or os.path.exists('config'):
+    # 导入用户自定义配置
+    ConfigManager.load(import_module('config').Config)
+
+if Config.DEPLOY_MODE == 'gevent':
+    # 异步
+    from gevent import monkey
+    monkey.patch_all()
+
+
 import sys
 from logging.config import dictConfig
 from multiprocessing import Process, set_start_method
@@ -10,27 +24,30 @@ from flask import Flask, make_response, request, send_from_directory
 
 import api
 import server
-import server.init
 import web.index
 import web.login
 from core.constant import Constant
-from core.download import (UserDownload, get_only_3_song_ids,
-                           initialize_songfile)
-from core.error import ArcError
+from core.download import UserDownload
+from core.error import ArcError, NoAccess, RateLimit
+from core.init import FileChecker
 from core.sql import Connect
 from server.func import error_return
-from setting import Config
 
 app = Flask(__name__)
 
-# from werkzeug.middleware.proxy_fix import ProxyFix
-# app.wsgi_app = ProxyFix(app.wsgi_app, x_proto=1, x_host=1)
-# from flask_cors import CORS
-# CORS(app, supports_credentials=True)
+if Config.USE_PROXY_FIX:
+    # 代理修复
+    from werkzeug.middleware.proxy_fix import ProxyFix
+    app.wsgi_app = ProxyFix(app.wsgi_app, x_proto=1, x_host=1)
+if Config.USE_CORS:
+    # 服务端跨域
+    from flask_cors import CORS
+    CORS(app, supports_credentials=True)
 
 
 os.chdir(sys.path[0])  # 更改工作路径，以便于愉快使用相对路径
 
+
 app.config.from_mapping(SECRET_KEY=Config.SECRET_KEY)
 app.config['SESSION_TYPE'] = 'filesystem'
 app.register_blueprint(web.login.bp)
@@ -55,22 +72,23 @@ def favicon():
 
 @app.route('/download/<path:file_path>', methods=['GET'])  # 下载
 def download(file_path):
-    with Connect() as c:
+    with Connect(in_memory=True) as c:
         try:
             x = UserDownload(c)
             x.token = request.args.get('t')
             x.song_id, x.file_name = file_path.split('/', 1)
             x.select_for_check()
             if x.is_limited:
-                raise ArcError(
-                    'You have reached the download limit.', 903, status=403)
+                raise RateLimit('You have reached the download limit.', 903)
             if not x.is_valid:
-                raise ArcError('Expired token.', status=403)
-            x.insert_user_download()
-            # response = make_response()
-            # response.headers['Content-Type'] = 'application/octet-stream'
-            # response.headers['X-Accel-Redirect'] = '/nginx_download/' + file_path
-            # return response
+                raise NoAccess('Expired token.')
+            x.download_hit()
+            if Config.DOWNLOAD_USE_NGINX_X_ACCEL_REDIRECT:
+                # nginx X-Accel-Redirect
+                response = make_response()
+                response.headers['Content-Type'] = 'application/octet-stream'
+                response.headers['X-Accel-Redirect'] = Config.NGINX_X_ACCEL_REDIRECT_PREFIX + file_path
+                return response
             return send_from_directory(Constant.SONG_FILE_FOLDER_PATH, file_path, as_attachment=True, conditional=True)
         except ArcError as e:
             if Config.ALLOW_WARNING_LOG:
@@ -79,10 +97,29 @@ def download(file_path):
     return error_return()
 
 
+if Config.DEPLOY_MODE == 'waitress':
+    # 给waitress加个日志
+    @app.after_request
+    def after_request(response):
+        app.logger.info(
+            f'{request.remote_addr} - - {request.method} {request.path} {response.status_code}')
+        return response
+
+
 def tcp_server_run():
-    if False:
+    if Config.DEPLOY_MODE == 'gevent':
+        # 异步 gevent WSGI server
+        host_port = (Config.HOST, Config.PORT)
+        app.logger.info('Running gevent WSGI server... (%s:%s)' % host_port)
         from gevent.pywsgi import WSGIServer
-        WSGIServer(("127.0.0.1", 5000), app).serve_forever()
+        WSGIServer(host_port, app, log=app.logger).serve_forever()
+    elif Config.DEPLOY_MODE == 'waitress':
+        # waitress WSGI server
+        import logging
+        from waitress import serve
+        logger = logging.getLogger('waitress')
+        logger.setLevel(logging.INFO)
+        serve(app, host=Config.HOST, port=Config.PORT)
     else:
         if Config.SSL_CERT and Config.SSL_KEY:
             app.run(Config.HOST, Config.PORT, ssl_context=(
@@ -91,6 +128,18 @@ def tcp_server_run():
             app.run(Config.HOST, Config.PORT)
 
 
+def generate_log_file_dict(level: str, filename: str) -> dict:
+    return {
+        "class": "logging.handlers.RotatingFileHandler",
+        "maxBytes": 1024 * 1024,
+        "backupCount": 1,
+        "encoding": "utf-8",
+        "level": level,
+        "formatter": "default",
+        "filename": filename
+    }
+
+
 def main():
     log_dict = {
         'version': 1,
@@ -104,15 +153,7 @@ def main():
                 'stream': 'ext://flask.logging.wsgi_errors_stream',
                 'formatter': 'default'
             },
-            "error_file": {
-                "class": "logging.handlers.RotatingFileHandler",
-                "maxBytes": 1024 * 1024,
-                "backupCount": 1,
-                "encoding": "utf-8",
-                "level": "ERROR",
-                "formatter": "default",
-                "filename": "./log/error.log"
-            }
+            "error_file": generate_log_file_dict('ERROR', './log/error.log')
         },
         'formatters': {
             'default': {
@@ -122,51 +163,30 @@ def main():
     }
     if Config.ALLOW_INFO_LOG:
         log_dict['root']['handlers'].append('info_file')
-        log_dict['handlers']['info_file'] = {
-            "class": "logging.handlers.RotatingFileHandler",
-            "maxBytes": 1024 * 1024,
-            "backupCount": 1,
-            "encoding": "utf-8",
-            "level": "INFO",
-            "formatter": "default",
-            "filename": "./log/info.log"
-        }
+        log_dict['handlers']['info_file'] = generate_log_file_dict(
+            'INFO', './log/info.log')
     if Config.ALLOW_WARNING_LOG:
         log_dict['root']['handlers'].append('warning_file')
-        log_dict['handlers']['warning_file'] = {
-            "class": "logging.handlers.RotatingFileHandler",
-            "maxBytes": 1024 * 1024,
-            "backupCount": 1,
-            "encoding": "utf-8",
-            "level": "WARNING",
-            "formatter": "default",
-            "filename": "./log/warning.log"
-        }
+        log_dict['handlers']['warning_file'] = generate_log_file_dict(
+            'WARNING', './log/warning.log')
 
     dictConfig(log_dict)
 
-    if not server.init.check_before_run(app):
-        app.logger.error('Something wrong. The server will not run.')
+    Connect.logger = app.logger
+    if not FileChecker(app.logger).check_before_run():
+        app.logger.error('Some errors occurred. The server will not run.')
         input('Press ENTER key to exit.')
         sys.exit()
 
-    app.logger.info("Start to initialize song data...")
-    try:
-        initialize_songfile()
-        get_only_3_song_ids()
-        app.logger.info('Complete!')
-    except:
-        app.logger.warning('Initialization error!')
-
     if Config.LINKPLAY_HOST and Config.SET_LINKPLAY_SERVER_AS_SUB_PROCESS:
         from linkplay_server import link_play
         process = [Process(target=link_play, args=(
             Config.LINKPLAY_HOST, int(Config.LINKPLAY_UDP_PORT), int(Config.LINKPLAY_TCP_PORT)))]
         [p.start() for p in process]
-        app.logger.info("Link Play UDP server is running on " +
-                        Config.LINKPLAY_HOST + ':' + str(Config.LINKPLAY_UDP_PORT) + " ...")
-        app.logger.info("Link Play TCP server is running on " +
-                        Config.LINKPLAY_HOST + ':' + str(Config.LINKPLAY_TCP_PORT) + " ...")
+        app.logger.info(
+            f"Link Play UDP server is running on {Config.LINKPLAY_HOST}:{Config.LINKPLAY_UDP_PORT} ...")
+        app.logger.info(
+            f"Link Play TCP server is running on {Config.LINKPLAY_HOST}:{Config.LINKPLAY_TCP_PORT} ...")
         tcp_server_run()
         [p.join() for p in process]
     else:

latest version/requirements.txt

@@ -1,2 +1,3 @@
 flask>=2.0.2
-cryptography>=35.0.0
+cryptography>=3.0.0
+limits>=2.7.0

latest version/run.bat

@@ -1,3 +1,4 @@
 cd /d %~dp0
 :: Develop server
-python -B main.py
+python -B main.py
+pause

latest version/server/__init__.py

@@ -1,5 +1,5 @@
+from core.config_manager import Config
 from flask import Blueprint
-from setting import Config
 
 from . import (auth, course, friend, multiplayer, others, present, purchase,
                score, user, world)

latest version/server/arcscore.py

@@ -1,56 +0,0 @@
-from core.sql import Connect
-
-
-def calculate_rating(defnum, score):
-    # 计算rating
-    if score >= 10000000:
-        ptt = defnum + 2
-    elif score < 9800000:
-        ptt = defnum + (score-9500000) / 300000
-        if ptt < 0 and defnum != -10:
-            ptt = 0
-    else:
-        ptt = defnum + 1 + (score-9800000) / 200000
-
-    return ptt
-
-
-def refresh_all_score_rating():
-    # 刷新所有best成绩的rating
-    error = 'Unknown error.'
-
-    with Connect() as c:
-        c.execute(
-            '''select song_id, rating_pst, rating_prs, rating_ftr, rating_byn from chart''')
-        x = c.fetchall()
-
-    if x:
-        song_list = [i[0] for i in x]
-        with Connect() as c:
-            c.execute('''update best_score set rating=0 where song_id not in ({0})'''.format(
-                ','.join(['?']*len(song_list))), tuple(song_list))
-            for i in x:
-                for j in range(0, 4):
-                    defnum = -10  # 没在库里的全部当做定数-10
-                    if i is not None:
-                        defnum = float(i[j+1]) / 10
-                        if defnum <= 0:
-                            defnum = -10  # 缺少难度的当做定数-10
-
-                    c.execute('''select user_id, score from best_score where song_id=:a and difficulty=:b''', {
-                              'a': i[0], 'b': j})
-                    y = c.fetchall()
-                    if y:
-                        for k in y:
-                            ptt = calculate_rating(defnum, k[1])
-                            if ptt < 0:
-                                ptt = 0
-
-                            c.execute('''update best_score set rating=:a where user_id=:b and song_id=:c and difficulty=:d''', {
-                                      'a': ptt, 'b': k[0], 'c': i[0], 'd': j})
-            error = None
-
-    else:
-        error = 'No song data.'
-
-    return error

latest version/server/auth.py

@@ -1,11 +1,11 @@
 import base64
 from functools import wraps
 
+from core.config_manager import Config
 from core.error import ArcError, NoAccess
 from core.sql import Connect
 from core.user import UserAuth, UserLogin
-from flask import Blueprint, jsonify, request
-from setting import Config
+from flask import Blueprint, g, jsonify, request
 
 from .func import arc_try, error_return
 
@@ -54,11 +54,11 @@ def auth_required(request):
                 try:
                     user = UserAuth(c)
                     user.token = headers['Authorization'][7:]
-                    return view(user.token_get_id(), *args, **kwargs)
+                    user_id = user.token_get_id()
+                    g.user = user
                 except ArcError as e:
                     return error_return(e)
-
-            return error_return()
+            return view(user_id, *args, **kwargs)
 
         return wrapped_view
     return decorator

latest version/server/course.py

@@ -1,6 +1,5 @@
 from core.constant import Constant
 from core.course import UserCourseList
-from core.error import ArcError
 from core.item import ItemCore
 from core.sql import Connect
 from core.user import UserOnline

latest version/server/func.py

@@ -1,9 +1,9 @@
 from functools import wraps
 from traceback import format_exc
 
+from core.config_manager import Config
 from core.error import ArcError
-from flask import current_app, jsonify
-from setting import Config
+from flask import current_app, g, jsonify
 
 default_error = ArcError('Unknown Error', status=500)
 
@@ -83,6 +83,9 @@ def arc_try(view):
         except ArcError as e:
             if Config.ALLOW_WARNING_LOG:
                 current_app.logger.warning(format_exc())
+            user = g.get("user", None)
+            current_app.logger.warning(
+                f'{user.user_id if user is not None else ""} - {e.error_code}|{e.api_error_code}: {e}')
             return error_return(e)
 
     return wrapped_view

latest version/server/init.py

@@ -1,94 +0,0 @@
-import os
-from shutil import copy, copy2
-from core.sql import Connect
-from database.database_initialize import main, ARCAEA_SERVER_VERSION
-from web.system import update_database
-
-
-def try_rename(path, new_path):
-    # 尝试重命名文件，并尝试避免命名冲突，返回最终路径
-    final_path = new_path
-    if os.path.exists(new_path):
-        i = 1
-        while os.path.exists(new_path + str(i)):
-            i += 1
-        os.rename(path, new_path + str(i))
-        final_path = new_path + str(i)
-    else:
-        os.rename(path, new_path)
-
-    return final_path
-
-
-def check_before_run(app):
-    # 运行前检查关键文件，返回布尔值，其实是因为有人经常忘了
-
-    f = True
-
-    if not os.path.exists('setting.py'):
-        app.logger.warning('File `setting.py` is missing.')
-        f = False
-
-    if not os.path.exists('database'):
-        app.logger.warning('Folder `database` is missing.')
-        f = False
-
-    if not os.path.exists('database/songs'):
-        app.logger.warning('Folder `database/songs` is missing.')
-        f = False
-
-    if not os.path.exists('database/arcaea_database.db'):
-        app.logger.warning('File `database/arcaea_database.db` is missing.')
-        f = False
-        try:
-            app.logger.info(
-                'Try to new the file `database/arcaea_database.db`.')
-            main('./database/')
-            app.logger.info(
-                'Success to new the file `database/arcaea_database.db`.')
-            f = True
-        except:
-            app.logger.warning(
-                'Fail to new the file `database/arcaea_database.db`.')
-
-    else:
-        with Connect() as c:
-            try:
-                c.execute('''select value from config where id="version"''')
-                x = c.fetchone()
-            except:
-                x = None
-
-        # 数据库自动更新，不强求
-        if not x or x[0] != ARCAEA_SERVER_VERSION:
-            app.logger.warning(
-                'Maybe the file `database/arcaea_database.db` is an old version.')
-            try:
-                app.logger.info(
-                    'Try to update the file `database/arcaea_database.db`.')
-
-                path = try_rename('database/arcaea_database.db',
-                                  'database/arcaea_database.db.bak')
-
-                try:
-                    copy2(path, 'database/arcaea_database.db')
-                except:
-                    copy(path, 'database/arcaea_database.db')
-
-                if os.path.isfile("database/old_arcaea_database.db"):
-                    os.remove('database/old_arcaea_database.db')
-
-                try_rename('database/arcaea_database.db',
-                           'database/old_arcaea_database.db')
-
-                main('./database/')
-                update_database()
-
-                app.logger.info(
-                    'Success to update the file `database/arcaea_database.db`.')
-
-            except:
-                app.logger.warning(
-                    'Fail to update the file `database/arcaea_database.db`.')
-
-    return f

latest version/server/multiplayer.py

@@ -1,8 +1,8 @@
+from core.config_manager import Config
 from core.error import ArcError
 from core.linkplay import Player, RemoteMultiPlayer, Room
 from core.sql import Connect
 from flask import Blueprint, request
-from setting import Config
 
 from .auth import auth_required
 from .func import arc_try, success_return

latest version/server/others.py

@@ -2,7 +2,7 @@ import json
 from urllib.parse import parse_qs, urlparse
 
 from core.download import DownloadList
-from core.error import ArcError
+from core.error import RateLimit
 from core.sql import Connect
 from core.system import GameInfo
 from core.user import UserOnline
@@ -12,7 +12,7 @@ from werkzeug.datastructures import ImmutableMultiDict
 from .auth import auth_required
 from .func import arc_try, error_return, success_return
 from .present import present_info
-from .purchase import bundle_pack, bundle_bundle
+from .purchase import bundle_bundle, bundle_pack
 from .score import song_score_friend
 from .user import user_me
 from .world import world_all
@@ -29,13 +29,12 @@ def game_info():
 @auth_required(request)
 @arc_try
 def download_song(user_id):
-    with Connect() as c:
-        x = DownloadList(c, UserOnline(c, user_id))
+    with Connect(in_memory=True) as c:
+        x = DownloadList(c, UserOnline(None, user_id))
         x.song_ids = request.args.getlist('sid')
         x.url_flag = json.loads(request.args.get('url', 'true'))
-        x.clear_user_download()
-        if x.is_limited and x.url_flag:
-            raise ArcError('You have reached the download limit.', 903)
+        if x.url_flag and x.is_limited:
+            raise RateLimit('You have reached the download limit.', 903)
 
         x.add_songs()
         return success_return(x.urls)

latest version/server/present.py

@@ -1,11 +1,10 @@
-from core.error import ArcError
 from core.present import UserPresent, UserPresentList
 from core.sql import Connect
 from core.user import UserOnline
 from flask import Blueprint, request
 
 from .auth import auth_required
-from .func import arc_try, error_return, success_return
+from .func import arc_try, success_return
 
 bp = Blueprint('present', __name__, url_prefix='/present')
 

latest version/server/user.py

@@ -1,11 +1,11 @@
 from core.character import UserCharacter
+from core.config_manager import Config
 from core.error import ArcError, NoAccess
 from core.item import ItemCore
 from core.save import SaveData
 from core.sql import Connect
 from core.user import User, UserLogin, UserOnline, UserRegister
 from flask import Blueprint, request
-from setting import Config
 
 from .auth import auth_required
 from .func import arc_try, success_return

latest version/web/index.py

@@ -1,8 +1,8 @@
 import os
 import time
 
-import server.arcscore
-from core.download import initialize_songfile, get_only_3_song_ids
+from core.init import FileChecker
+from core.operation import RefreshAllScoreRating, RefreshSongFileCache
 from core.rank import RankList
 from core.sql import Connect
 from flask import Blueprint, flash, redirect, render_template, request, url_for
@@ -270,7 +270,8 @@ def update_database():
             file.save(os.path.join(UPLOAD_FOLDER, filename))
             flash('上传成功 Success upload.')
             try:
-                web.system.update_database()
+                FileChecker.update_database(
+                    os.path.join(UPLOAD_FOLDER, filename))
                 flash('数据更新成功 Success update data.')
             except:
                 flash('数据更新失败 Cannot update data.')
@@ -288,9 +289,7 @@ def update_database():
 def update_song_hash():
     # 更新数据库内谱面文件hash值
     try:
-        get_only_3_song_ids.cache_clear()
-        get_only_3_song_ids()
-        initialize_songfile()
+        RefreshSongFileCache().run()
         flash('数据刷新成功 Success refresh data.')
     except:
         flash('Something error!')
@@ -301,11 +300,8 @@ def update_song_hash():
 @login_required
 def update_song_rating():
     # 更新所有分数的rating
-    error = server.arcscore.refresh_all_score_rating()
-    if error:
-        flash(error)
-    else:
-        flash('数据刷新成功 Success refresh data.')
+    RefreshAllScoreRating().run()
+    flash('数据刷新成功 Success refresh data.')
     return render_template('web/updatedatabase.html')
 
 
@@ -425,7 +421,7 @@ def all_character():
 def change_character():
     # 修改角色数据
     skill_ids = ['No_skill', 'gauge_easy', 'note_mirror', 'gauge_hard', 'frag_plus_10_pack_stellights', 'gauge_easy|frag_plus_15_pst&prs', 'gauge_hard|fail_frag_minus_100', 'frag_plus_5_side_light', 'visual_hide_hp', 'frag_plus_5_side_conflict', 'challenge_fullcombo_0gauge', 'gauge_overflow', 'gauge_easy|note_mirror', 'note_mirror', 'visual_tomato_pack_tonesphere',
-                 'frag_rng_ayu', 'gaugestart_30|gaugegain_70', 'combo_100-frag_1', 'audio_gcemptyhit_pack_groovecoaster', 'gauge_saya', 'gauge_chuni', 'kantandeshou', 'gauge_haruna', 'frags_nono', 'gauge_pandora', 'gauge_regulus', 'omatsuri_daynight', 'sometimes(note_mirror|frag_plus_5)', 'scoreclear_aa|visual_scoregauge', 'gauge_tempest', 'gauge_hard', 'gauge_ilith_summer', 'frags_kou', 'visual_ink', 'shirabe_entry_fee', 'frags_yume', 'note_mirror|visual_hide_far', 'frags_ongeki', 'gauge_areus', 'gauge_seele', 'gauge_isabelle', 'gauge_exhaustion', 'skill_lagrange', 'gauge_safe_10', 'frags_nami', 'skill_elizabeth', 'skill_lily', 'skill_kanae_midsummer', 'eto_uncap', 'luna_uncap', 'frags_preferred_song', 'visual_ghost_skynotes', 'ayu_uncap', 'skill_vita', 'skill_fatalis', 'skill_reunion', 'frags_ongeki_slash', 'frags_ongeki_hard']
+                 'frag_rng_ayu', 'gaugestart_30|gaugegain_70', 'combo_100-frag_1', 'audio_gcemptyhit_pack_groovecoaster', 'gauge_saya', 'gauge_chuni', 'kantandeshou', 'gauge_haruna', 'frags_nono', 'gauge_pandora', 'gauge_regulus', 'omatsuri_daynight', 'sometimes(note_mirror|frag_plus_5)', 'scoreclear_aa|visual_scoregauge', 'gauge_tempest', 'gauge_hard', 'gauge_ilith_summer', 'frags_kou', 'visual_ink', 'shirabe_entry_fee', 'frags_yume', 'note_mirror|visual_hide_far', 'frags_ongeki', 'gauge_areus', 'gauge_seele', 'gauge_isabelle', 'gauge_exhaustion', 'skill_lagrange', 'gauge_safe_10', 'frags_nami', 'skill_elizabeth', 'skill_lily', 'skill_kanae_midsummer', 'eto_uncap', 'luna_uncap', 'frags_preferred_song', 'visual_ghost_skynotes', 'ayu_uncap', 'skill_vita', 'skill_fatalis', 'skill_reunion', 'frags_ongeki_slash', 'frags_ongeki_hard', 'skill_amane']
     return render_template('web/changechar.html', skill_ids=skill_ids)
 
 
@@ -699,7 +695,7 @@ def change_item():
             c.execute(
                 '''select exists(select * from item where item_id=:a and type=:b)''', {'a': item_id, 'b': item_type})
             if c.fetchone() == (0,):
-                c.execute('''insert into item values(?,?,?,'')''',
+                c.execute('''insert into item values(?,?,?)''',
                           (item_id, item_type, is_available))
                 flash('物品添加成功 Successfully add the item.')
             else:

latest version/web/login.py

@@ -1,10 +1,10 @@
-#import sqlite3
-from flask import (Blueprint, flash, g, redirect,
-                   render_template, request, session, url_for)
 import functools
-from setting import Config
 import hashlib
 
+from core.config_manager import Config
+from flask import (Blueprint, flash, g, redirect, render_template, request,
+                   session, url_for)
+
 bp = Blueprint('login', __name__, url_prefix='/web')
 
 

latest version/web/system.py

@@ -1,9 +1,8 @@
-import os
-from core.sql import Connect
-import time
 import hashlib
+import time
 from random import Random
-from setting import Config
+
+from core.sql import Connect
 
 
 def int2b(x):
@@ -25,139 +24,6 @@ def random_str(randomlength=10):
     return s
 
 
-def get_table_info(c, table_name):
-    # 得到表结构，返回主键列表和字段名列表
-    pk = []
-    name = []
-
-    c.execute('''pragma table_info ("'''+table_name+'''")''')
-    x = c.fetchall()
-    if x:
-        for i in x:
-            name.append(i[1])
-            if i[5] != 0:
-                pk.append(i[1])
-
-    return pk, name
-
-
-def get_sql_select_table(table_name, get_field, where_field=[], where_value=[]):
-    # sql语句拼接，select ... from ... where ...
-    sql = 'select '
-    sql_dict = {}
-    if len(get_field) >= 2:
-        sql += get_field[0]
-        for i in range(1, len(get_field)):
-            sql += ',' + get_field[i]
-        sql += ' from ' + table_name
-    elif len(get_field) == 1:
-        sql += get_field[0] + ' from ' + table_name
-    else:
-        sql += '* from ' + table_name
-
-    if where_field and where_value:
-        sql += ' where '
-        sql += where_field[0] + '=:' + where_field[0]
-        sql_dict[where_field[0]] = where_value[0]
-        if len(where_field) >= 1:
-            for i in range(1, len(where_field)):
-                sql_dict[where_field[i]] = where_value[i]
-                sql += ' and ' + where_field[i] + '=:' + where_field[i]
-
-    sql += ' order by rowid'
-    return sql, sql_dict
-
-
-def get_sql_insert_table(table_name, field, value):
-    # sql语句拼接，insert into ...(...) values(...)
-    sql = 'insert into ' + table_name + '('
-    sql_dict = {}
-    sql2 = ''
-    if len(field) >= 2:
-        sql += field[0]
-        sql2 += ':' + field[0]
-        sql_dict[field[0]] = value[0]
-        for i in range(1, len(field)):
-            sql += ',' + field[i]
-            sql2 += ', :' + field[i]
-            sql_dict[field[i]] = value[i]
-
-        sql += ') values('
-
-    elif len(field) == 1:
-        sql += field[0] + ') values('
-        sql2 += ':' + field[0]
-        sql_dict[field[0]] = value[0]
-
-    else:
-        return 'error', {}
-
-    sql += sql2 + ')'
-
-    return sql, sql_dict
-
-
-def get_sql_delete_table(table_name, where_field=[], where_value=[]):
-    # sql语句拼接，delete from ... where ...
-    sql = 'delete from ' + table_name
-    sql_dict = {}
-
-    if where_field and where_value:
-        sql += ' where '
-        sql += where_field[0] + '=:' + where_field[0]
-        sql_dict[where_field[0]] = where_value[0]
-        if len(where_field) >= 1:
-            for i in range(1, len(where_field)):
-                sql_dict[where_field[i]] = where_value[i]
-                sql += ' and ' + where_field[i] + '=:' + where_field[i]
-
-    return sql, sql_dict
-
-
-def update_one_table(c1, c2, table_name):
-    # 从c1向c2更新数据表，c1中存在的信息不变
-    c1.execute(
-        '''select * from sqlite_master where type = 'table' and name = :a''', {'a': table_name})
-    c2.execute(
-        '''select * from sqlite_master where type = 'table' and name = :a''', {'a': table_name})
-    if not c1.fetchone() or not c2.fetchone():
-        return 'error'
-
-    db1_pk, db1_name = get_table_info(c1, table_name)
-    db2_pk, db2_name = get_table_info(c2, table_name)
-    if db1_pk != db2_pk:
-        return 'error'
-
-    field = []
-    for i in db1_name:
-        if i in db2_name:
-            field.append(i)
-
-    sql, sql_dict = get_sql_select_table(table_name, db1_pk)
-    c1.execute(sql)
-    x = c1.fetchall()
-    sql, sql_dict = get_sql_select_table(table_name, field)
-    c1.execute(sql)
-    y = c1.fetchall()
-    if x:
-        for i in range(0, len(x)):
-            sql, sql_dict = get_sql_select_table(
-                table_name, [], db1_pk, list(x[i]))
-            sql = 'select exists(' + sql + ')'
-            c2.execute(sql, sql_dict)
-
-            if c2.fetchone() == (1,):  # 如果c2里存在，先删除
-                sql, sql_dict = get_sql_delete_table(
-                    table_name, db1_pk, list(x[i]))
-                c2.execute(sql, sql_dict)
-
-            sql, sql_dict = get_sql_insert_table(
-                table_name, field, list(y[i]))
-            c2.execute(sql, sql_dict)
-
-    return None
-
-
 def update_user_char(c):
     # 用character数据更新user_char
     c.execute('''select character_id, max_level, is_uncapped from character''')
@@ -174,64 +40,6 @@ def update_user_char(c):
                           (j[0], i[0], i[1], exp, i[2], 0))
 
 
-def update_user_epilogue(c):
-    c.execute('''select user_id from user''')
-    x = c.fetchall()
-    for i in x:
-        c.execute(
-            '''select exists(select * from user_item where user_id=? and item_id=? and type='pack')''', (i[0], 'epilogue'))
-        if c.fetchone() == (0,):
-            c.execute('''insert into user_item values(?,?,'pack',1)''',
-                      (i[0], 'epilogue'))
-
-
-def update_database():
-    # 将old数据库不存在数据加入到新数据库上，并删除old数据库
-    # 对于arcaea_datebase.db，更新一些表，并用character数据更新user_char_full
-    if os.path.isfile("database/old_arcaea_database.db") and os.path.isfile("database/arcaea_database.db"):
-        with Connect('./database/old_arcaea_database.db') as c1:
-            with Connect() as c2:
-
-                update_one_table(c1, c2, 'user')
-                update_one_table(c1, c2, 'friend')
-                update_one_table(c1, c2, 'best_score')
-                update_one_table(c1, c2, 'recent30')
-                update_one_table(c1, c2, 'user_world')
-                update_one_table(c1, c2, 'item')
-                update_one_table(c1, c2, 'user_item')
-                update_one_table(c1, c2, 'purchase')
-                update_one_table(c1, c2, 'purchase_item')
-                update_one_table(c1, c2, 'user_save')
-                update_one_table(c1, c2, 'login')
-                update_one_table(c1, c2, 'present')
-                update_one_table(c1, c2, 'user_present')
-                update_one_table(c1, c2, 'present_item')
-                update_one_table(c1, c2, 'redeem')
-                update_one_table(c1, c2, 'user_redeem')
-                update_one_table(c1, c2, 'redeem_item')
-                update_one_table(c1, c2, 'role')
-                update_one_table(c1, c2, 'user_role')
-                update_one_table(c1, c2, 'power')
-                update_one_table(c1, c2, 'role_power')
-                update_one_table(c1, c2, 'api_login')
-                update_one_table(c1, c2, 'chart')
-                update_one_table(c1, c2, 'user_course')
-                update_one_table(c1, c2, 'course')
-                update_one_table(c1, c2, 'course_item')
-                update_one_table(c1, c2, 'course_chart')
-                update_one_table(c1, c2, 'course_requirement')
-
-                update_one_table(c1, c2, 'user_char')
-
-                if not Config.UPDATE_WITH_NEW_CHARACTER_DATA:
-                    update_one_table(c1, c2, 'character')
-
-                update_user_char(c2)  # 更新user_char_full
-                update_user_epilogue(c2)  # 更新user的epilogue
-
-        os.remove('database/old_arcaea_database.db')
-
-
 def unlock_all_user_item(c):
     # 解锁所有用户购买