Merge pull request #75 from Lost-MSth/dev

Update to v2.10.1

.gitignore

@@ -1,5 +1,20 @@
+# log files
 *.log
+*.log.*
+
+# SSL cert
+*.pem
+*.key
+
+# sqlite3 database
+*.db
+*.db.bak*
+
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]
-*$py.class
+*$py.class
+
+# setting/config files
+latest version/config/
+latest version/config.py

README.md

@@ -67,39 +67,36 @@ It is just so interesting. What it can do is under exploration.
 > Tips: When updating, please keep the original database in case of data loss. 
 
 
-### Version 2.10.0
+### Version 2.10.1
-- 适用于Arcaea 4.1.0版本 For Arcaea 4.1.0
+- 适用于Arcaea 4.1.4版本 For Arcaea 4.1.4
-- 新搭档 **咲姬** 已解锁 Unlock the character **Saki**.
+- 新搭档 **天音** 已解锁 Unlock the character **Amane**.
-- 新搭档 **刹那** 已解锁 Unlock the character **Setsuna**.
+- 为**天音**技能提供支持 Add support for the skill of **Amane**.
-- 完善了日志系统 Improve the log system.
+- 现在配置文件可以是含有部分选项的文件或模块 At present the setting file can be a module or a file with some of options.
-- 现在可以利用`songlist`确保`3.aff`以外文件不被下载 Now you can use `songlist` to ensure that files other than `3.aff` should not be downloaded. [#60](https://github.com/Lost-MSth/Arcaea-server/issues/60)
+- 添加`waitress`和`gevent`的部署方案支持，并支持日志记录 Add deployment mode `waitress` and `gevent`, and add support for the info log recording of them.
-- 适配v4.0.0以下版本的客户端云存档 Ensure that the clients under v4.0.0 can upload the cloud save.
+- 为`songlist`添加解析器以指定可下载的文件 Add a parser for `songlist` to specify downloadable files.
-- 优化数据库索引 Optimize database indices.
+- 重构数据库初始化和数据迁移部分 Code refactoring for database initialization and migration.
-- 尝试确保LinkPlay服务器的线程安全，现在此功能将作为独立服务端 Try to ensure thread safety in LinkPlay server. Now this function will be served as an independent server.
+- 限制用户下载频率将使用第三方限制器，替代数据库 Add a custom limiter and use it for limiting users' download rate instead of using database.
-- 对API接口的分数列表添加歌曲名 Add song names for getting the score list in API.
+  > 现在需要`limits`模块  
-- 为下载错误增添HTTP状态码 Add HTTP status code when meeting download error.
+  > Now `limits` module is required.
-
+- 为登录和API登录添加限制器 Add limiter for login and API login.
-- 修复iOS客户端因世界模式地图数据闪退的问题 Fix a bug when world maps' data don't have some unnecessary parts the client of iOS may break down.
+- `sqlite3`数据库调整为WAL模式并增大缓存 Change journal mode to WAL and enlarge cache size for `sqlite3` database.
-- 修复API接口无数据`GET`请求导致报错的问题 Fix a bug that `GET` requests without data will report an error in API. [#50](https://github.com/Lost-MSth/Arcaea-server/issues/50)
+- 将下载token放入内存中而不是文件数据库中 Put download token in memory database instead of filesystem database.
-- 修复`aggregate`请求无法捕获内部错误的问题 Fix a bug that `aggregate` requests will get an error when the inner function raises an error.
+- 加速`best_score`表多次查询，表现为歌曲排行榜查询性能提升 Accelerate multiple querying in `best_score` table, which results in performance improvement of song ranklist query.
-- 修复因错误设置主键导致课程模式谱面无法相同的问题 Fix a bug that the charts of a course cannot be the same because of the incorrect primary keys.
+- 优化歌曲下载部分 Make some optimization for downloading songs.
-- 修复无谱面数据时世界排名分计算出错的问题 Fix a bug that global ranking scores cannot be calculated if there are no chart in the database. [#61](https://github.com/Lost-MSth/Arcaea-server/issues/61)
+- **修复更新recent 10时可能出现的死循环问题 Fix a bug that there is a endless loop in calculating recent 10 updating.** (due to 6fcca179182775615115cdb255b3a8223831a8a0)
-- 修复条件满足但隐藏段位依然无法解锁的问题 Fix a bug that the hidden courses cannot appear even if their requirements are satisfied.
+- 修复课题模式成绩没有rating的问题 Fix a bug that scores in course mode cannot calculate rating.
-- 修复Finale挑战中某些无法解锁的问题 Fix a bug that something of the finale challenge cannot be unlocked.
+- 修正搭档数值 Fix a character's value.
-- 修复用户物品数量无法为0的问题，此问题导致了一些购买操作异常 Fix a bug that the users' items will not be zero, which will disturb some purchase operations.
+- 邮箱长度最大限制提升到64 Change the email max length to 64.
-- 修复角色等级能超过最大等级的问题 Fix a bug that the level of the character can exceed the max level.
+- 新增API接口来获取用户身份与权限 Add a method of API for getting users' roles and powers.
-- 修复使用`以太之滴`升级角色时应答不正确的问题 Fix a bug that the response is incorrect when upgrading the characters by `generic core`.
+- 新增API接口来修改用户信息 Add a method of API to change the user's info.
-- 修复`源韵强化`数值显示不正确的问题 Fix a bug that the `prog boost` shows the incorrect value.
+- 为API的`GET`请求添加`query`参数支持 Add support for the `query` param in API's `GET` requests.
-- 修复世界模式奖励可能被重复发放的问题 Fix a bug that the rewards can be get repeatedly in World Mode.
+- 修复API的`best30`接口曲目无数据导致出错的问题 Fix a bug that `best30` of API cannot have scores whose songs are not in database.
-- 修复世界Boss的第二管血量无法削减的问题 Fix a bug that second tube of blood of the world boss won't change.
+- 修复API的`recent30`接口用户成绩数量不足导致出错的问题 Fix a bug that users with no recent scores cannot get `recent30` via API.
-- 修复我的排名显示不正确的问题 Fix a bug that `my rank` doesn't work correctly.
-- 修复在歌曲结束后无法及时轮换房主的问题 Fix a bug that the room host will be changed late when finishing a song.
-
 
 ## 运行环境与依赖 Running environment and requirements
 - Windows/Linux/Mac OS/Android...
-- Python 3
+- Python >= 3.6
-- Flask module >= 2.0, Cryptography module >= 35.0.0
+- Flask module >= 2.0, Cryptography module >= 3.0.0, limits >= 2.7.0
 - Charles, IDA, proxy app... (optional)
 
 <!--

latest version/api/api_auth.py

@@ -1,11 +1,13 @@
 from functools import wraps
 from traceback import format_exc
+from base64 import b64decode
+from json import loads
 
 from core.api_user import APIUser
+from core.config_manager import Config
 from core.error import ArcError, NoAccess, PostError
 from core.sql import Connect
 from flask import current_app
-from setting import Config
 
 from .api_code import error_return
 
@@ -25,20 +27,17 @@ def role_required(request, powers=[]):
                 return error_return(PostError('No token', api_error_code=-1), 401)
 
             user = APIUser()
-            if Config.API_TOKEN == request.headers['Token'] and Config.API_TOKEN != '':
+            with Connect() as c:
-                user.user_id = 0
+                user.c = c
-            elif powers == []:
+                if Config.API_TOKEN == request.headers['Token'] and Config.API_TOKEN != '':
-                # 无powers则非本地权限（API_TOKEN规定的）无法访问
+                    user.set_role_system()
-                return error_return(NoAccess('No permission', api_error_code=-1), 403)
+                else:
-            else:
-                with Connect() as c:
                     try:
-                        user.c = c
                         user.select_user_id_from_api_token(
                             request.headers['Token'])
                         user.select_role_and_powers()
 
-                        if not any([y in [x.power_name for x in user.role.powers] for y in powers]):
+                        if not any(user.role.has_power(y) for y in powers):
                             return error_return(NoAccess('No permission', api_error_code=-1), 403)
                     except ArcError as e:
                         return error_return(e, 401)
@@ -63,17 +62,24 @@ def request_json_handle(request, required_keys=[], optional_keys=[]):
         def wrapped_view(*args, **kwargs):
 
             data = {}
-            if not request.data:
+            if request.data:
-                return view(data, *args, **kwargs)
+                json_data = request.json
+            else:
+                if request.method == 'GET' and 'query' in request.args:
+                    # 处理axios没法GET传data的问题
+                    json_data = loads(
+                        b64decode(request.args['query']).decode())
+                else:
+                    json_data = {}
 
             for key in required_keys:
-                if key not in request.json:
+                if key not in json_data:
-                    return error_return(PostError('Missing parameter: ' + key, api_error_code=-100))
+                    return error_return(PostError(f'Missing parameter: {key}', api_error_code=-100))
-                data[key] = request.json[key]
+                data[key] = json_data[key]
 
             for key in optional_keys:
-                if key in request.json:
+                if key in json_data:
-                    data[key] = request.json[key]
+                    data[key] = json_data[key]
 
             return view(data, *args, **kwargs)
 

latest version/api/api_code.py

@@ -6,20 +6,23 @@ default_error = ArcError('Unknown Error')
 
 CODE_MSG = {
     0: '',
-    -1: 'See status code',
+    -1: 'See status code',  # 基础错误
     -2: 'No data',
-    -3: 'No data or user',
+    -3: 'No data or user',  # 不确定是无数据还是无用户
-    -4: 'No user_id',
+    -100: 'Invalid post data',  # 1xx数据错误
-    -100: 'Wrong post data',
+    -101: 'Invalid data type',
-    -101: 'Wrong data type',
+    -102: 'Invalid query parameter',
-    -102: 'Wrong query parameter',
+    -103: 'Invalid sort parameter',
-    -103: 'Wrong sort parameter',
+    -104: 'Invalid sort order parameter',
-    -104: 'Wrong sort order parameter',
+    -105: 'Invalid URL parameter',
-    -200: 'No permission',
+    -110: 'Invalid user_id',
+    -200: 'No permission',  # 2xx用户相关错误
     -201: 'Wrong username or password',
     -202: 'User is banned',
-    -203: 'Username exists',
+    -203: 'Too many login attempts',
-    -204: 'Email address exists',
+    -210: 'Username exists',
+    -211: 'Email address exists',
+    -212: 'User code exists',
     -999: 'Unknown error'
 }
 

latest version/api/songs.py

@@ -30,7 +30,7 @@ def songs_get(data, user):
     B = ['song_id', 'name', 'rating_pst',
          'rating_prs', 'rating_ftr', 'rating_byn']
     with Connect() as c:
-        query = Query(A, A, B).from_data(data)
+        query = Query(A, A, B).from_dict(data)
         x = Sql(c).select('chart', query=query)
         r = []
         for i in x:

latest version/api/users.py

@@ -1,7 +1,8 @@
 from core.error import InputError, NoAccess, NoData
 from core.score import Potential, UserScoreList
 from core.sql import Connect, Query, Sql
-from core.user import UserInfo, UserRegister
+from core.user import UserChanger, UserInfo, UserRegister
+from core.api_user import APIUser
 from flask import Blueprint, request
 
 from .api_auth import api_try, request_json_handle, role_required
@@ -15,7 +16,7 @@ bp = Blueprint('users', __name__, url_prefix='/users')
 @role_required(request, ['change'])
 @request_json_handle(request, ['name', 'password', 'email'])
 @api_try
-def users_post(data, _):
+def users_post(data, user):
     '''注册一个用户'''
     with Connect() as c:
         new_user = UserRegister(c)
@@ -36,7 +37,7 @@ def users_get(data, user):
     B = ['user_id', 'name', 'user_code', 'join_date',
          'rating_ptt', 'time_played', 'ticket', 'world_rank_score']
     with Connect() as c:
-        query = Query(A, A, B).from_data(data)
+        query = Query(A, A, B).from_dict(data)
         x = Sql(c).select('user', query=query)
         r = []
         for i in x:
@@ -50,7 +51,7 @@ def users_get(data, user):
             'name': x.name,
             'join_date': x.join_date,
             'user_code': x.user_code,
-            'rating_ptt': x.rating_ptt/100,
+            'rating_ptt': x.rating_ptt,
             'character_id': x.character.character_id,
             'is_char_uncapped': x.character.is_uncapped,
             'is_char_uncapped_override': x.character.is_uncapped_override,
@@ -65,9 +66,9 @@ def users_get(data, user):
 def users_user_get(user, user_id):
     '''查询用户信息'''
     if user_id <= 0:
-        return error_return(InputError(api_error_code=-4))
+        return error_return(InputError(api_error_code=-110))
     # 查别人需要select权限
-    if user_id != user.user_id and user.user_id != 0 and not user.role.has_power('select'):
+    if user_id != user.user_id and not user.role.has_power('select'):
         return error_return(NoAccess('No permission', api_error_code=-1), 403)
 
     with Connect() as c:
@@ -75,21 +76,56 @@ def users_user_get(user, user_id):
         return success_return(u.to_dict())
 
 
+@bp.route('/<int:user_id>', methods=['PUT'])
+@role_required(request, ['change'])
+@request_json_handle(request, optional_keys=['name', 'password', 'user_code', 'ticket', 'email'])
+@api_try
+def users_user_put(data, user, user_id):
+    '''修改一个用户'''
+    with Connect() as c:
+        u = UserChanger(c, user_id)
+        r = {}
+        r['user_id'] = user_id
+        if 'name' in data:
+            u.set_name(data['name'])
+            r['name'] = u.name
+        if 'password' in data:
+            u.set_password(data['password'])
+            r['password'] = u.hash_pwd
+        if 'email' in data:
+            u.set_email(data['email'])
+            r['email'] = u.email
+        if 'user_code' in data:
+            u.set_user_code(data['user_code'])
+            r['user_code'] = u.user_code
+        if 'ticket' in data:
+            if not isinstance(data['ticket'], int):
+                raise InputError('Ticket must be int')
+            u.ticket = data['ticket']
+            r['ticket'] = u.ticket
+        if r:
+            u.update_columns(d=r)
+        return success_return(r)
+
+
 @bp.route('/<int:user_id>/b30', methods=['GET'])
 @role_required(request, ['select', 'select_me'])
 @api_try
 def users_user_b30_get(user, user_id):
     '''查询用户b30'''
     if user_id <= 0:
-        return error_return(InputError(api_error_code=-4))
+        return error_return(InputError(api_error_code=-110))
     # 查别人需要select权限
-    if user_id != user.user_id and user.user_id != 0 and not user.role.has_power('select'):
+    if user_id != user.user_id and not user.role.has_power('select'):
         return error_return(NoAccess('No permission', api_error_code=-1), 403)
 
     with Connect() as c:
         x = UserScoreList(c, UserInfo(c, user_id))
         x.query.limit = 30
         x.select_from_user()
+        if not x.scores:
+            raise NoData(
+                f'No best30 data of user `{user_id}`', api_error_code=-3)
         x.select_song_name()
         r = x.to_dict_list()
         rating_sum = sum([i.rating for i in x.scores])
@@ -103,15 +139,18 @@ def users_user_b30_get(user, user_id):
 def users_user_best_get(data, user, user_id):
     '''查询用户所有best成绩'''
     if user_id <= 0:
-        return error_return(InputError(api_error_code=-4))
+        return error_return(InputError(api_error_code=-110))
     # 查别人需要select权限
-    if user_id != user.user_id and user.user_id != 0 and not user.role.has_power('select'):
+    if user_id != user.user_id and not user.role.has_power('select'):
         return error_return(NoAccess('No permission', api_error_code=-1), 403)
 
     with Connect() as c:
         x = UserScoreList(c, UserInfo(c, user_id))
-        x.query.from_data(data)
+        x.query.from_dict(data)
         x.select_from_user()
+        if not x.scores:
+            raise NoData(
+                f'No best score data of user `{user_id}`', api_error_code=-3)
         r = x.to_dict_list()
         return success_return({'user_id': user_id, 'data': r})
 
@@ -123,11 +162,32 @@ def users_user_r30_get(user, user_id):
     '''查询用户r30'''
 
     if user_id <= 0:
-        return error_return(InputError(api_error_code=-4))
+        return error_return(InputError(api_error_code=-110))
     # 查别人需要select权限
-    if user_id != user.user_id and user.user_id != 0 and not user.role.has_power('select'):
+    if user_id != user.user_id and not user.role.has_power('select'):
         return error_return(NoAccess('No permission', api_error_code=-1), 403)
 
     with Connect() as c:
         p = Potential(c, UserInfo(c, user_id))
         return success_return({'user_id': user_id, 'r10_ptt': p.recent_10 / 10, 'data': p.recent_30_to_dict_list()})
+
+
+@bp.route('/<int:user_id>/role', methods=['GET'])
+@role_required(request, ['select', 'select_me'])
+@api_try
+def users_user_role_get(user, user_id):
+    '''查询用户role和powers'''
+
+    if user_id <= 0:
+        return error_return(InputError(api_error_code=-110))
+
+    if user_id == user.user_id:
+        return success_return({'user_id': user.user_id, 'role': user.role.role_id, 'powers': [i.power_id for i in user.role.powers]})
+    # 查别人需要select权限
+    if not user.role.has_power('select'):
+        return error_return(NoAccess('No permission', api_error_code=-1), 403)
+
+    with Connect() as c:
+        x = APIUser(c, user_id)
+        x.select_role_and_powers()
+        return success_return({'user_id': x.user_id, 'role': x.role.role_id, 'powers': [i.power_id for i in x.role.powers]})

latest version/config.example.py

@@ -1,6 +1,7 @@
 class Config():
     '''
-    This is the setting file. You can change some parameters here.
+    This is the example setting file.
+    The user's setting file's name is `config.py`.
     '''
 
     '''
@@ -9,7 +10,7 @@ class Config():
     Host and port of your server
     '''
     HOST = '0.0.0.0'
-    PORT = '80'
+    PORT = 80
     '''
     --------------------
     '''

latest version/core/api_user.py

@@ -2,57 +2,49 @@ from hashlib import sha256
 from os import urandom
 from time import time
 
-from .error import NoAccess, NoData, UserBan
+from .config_manager import Config
+from .error import NoAccess, NoData, RateLimit, UserBan
+from .limiter import ArcLimiter
 from .user import UserOnline
 
 
 class Power:
     def __init__(self, c=None):
         self.c = c
-        self.power_id: int = None
+        self.power_id: str = None
-        self.power_name: str = None
         self.caption: str = None
 
     @classmethod
     def from_dict(cls, d: dict, c=None) -> 'Power':
         p = cls(c)
         p.power_id = d['power_id']
-        p.power_name = d['power_name']
         p.caption = d['caption']
         return p
 
-    def select_from_name(self, power_name: str) -> 'Power':
-        pass
-
 
 class Role:
     def __init__(self, c=None):
         self.c = c
-        self.role_id: int = None
+        self.role_id: str = None
-        self.role_name: str = None
         self.caption: str = None
 
         self.powers: list = None
 
-    def has_power(self, power_name: str) -> bool:
+    def has_power(self, power_id: str) -> bool:
         '''判断role是否有power'''
-        for i in self.powers:
+        return any(power_id == i.power_id for i in self.powers)
-            if i.power_name == power_name:
-                return True
-        return False
 
     def select_from_id(self, role_id: int = None) -> 'Role':
         '''用role_id查询role'''
         if role_id is not None:
             self.role_id = role_id
-        self.c.execute('''select role_name, caption from role where role_id = :a''',
+        self.c.execute('''select caption from role where role_id = :a''',
                        {'a': self.role_id})
         x = self.c.fetchone()
         if x is None:
             raise NoData('The role `%s` does not exist.' %
                          self.role_id, api_error_code=-200)
-        self.role_name = x[0]
+        self.caption = x[0]
-        self.caption = x[1]
         return self
 
     def select_powers(self) -> None:
@@ -63,10 +55,12 @@ class Role:
         x = self.c.fetchall()
         for i in x:
             self.powers.append(Power.from_dict(
-                {'power_id': i[0], 'power_name': i[1], 'caption': i[2]}, self.c))
+                {'power_id': i[0], 'caption': i[1]}, self.c))
 
 
 class APIUser(UserOnline):
+    limiter = ArcLimiter(Config.API_LOGIN_RATE_LIMIT, 'api_login')
+
     def __init__(self, c=None, user_id=None) -> None:
         super().__init__(c, user_id)
         self.api_token: str = None
@@ -74,6 +68,13 @@ class APIUser(UserOnline):
 
         self.ip: str = None
 
+    def set_role_system(self) -> None:
+        '''设置为最高权限用户，API接口'''
+        self.user_id = 0
+        self.role = Role(self.c)
+        self.role.role_id = 'system'
+        self.role.select_powers()
+
     def select_role(self) -> None:
         '''查询user的role'''
         self.c.execute('''select role_id from user_role where user_id = :a''',
@@ -82,10 +83,9 @@ class APIUser(UserOnline):
         self.role = Role(self.c)
         if x is None:
             # 默认role为user
-            self.role.role_id = 1
+            self.role.role_id = 'user'
         else:
-            self.role.role_id = int(x[0])
+            self.role.role_id = x[0]
-        self.role.select_from_id()
 
     def select_role_and_powers(self) -> None:
         '''查询user的role，以及role的powers'''
@@ -113,6 +113,9 @@ class APIUser(UserOnline):
             self.password = password
         if ip is not None:
             self.ip = ip
+        if not self.limiter.hit(name):
+            raise RateLimit('Too many login attempts', api_error_code=-205)
+
         self.c.execute('''select user_id, password from user where name = :a''', {
                        'a': self.name})
         x = self.c.fetchone()

latest version/core/character.py

@@ -1,6 +1,6 @@
-from setting import Config
+from .config_manager import Config
-from .error import ArcError, InputError, NoData, ItemNotEnough
 from .constant import Constant
+from .error import ArcError, InputError, ItemNotEnough, NoData
 from .item import Item, ItemCore
 
 

latest version/core/config_manager.py

@@ -0,0 +1,93 @@
+class Config:
+    '''
+    Default config
+    '''
+
+    HOST = '0.0.0.0'
+    PORT = 80
+
+    DEPLOY_MODE = 'flask_multithread'
+    USE_PROXY_FIX = False
+    USE_CORS = False
+
+    GAME_API_PREFIX = '/join/21'
+
+    ALLOW_APPVERSION = []  # list[str]
+
+    SET_LINKPLAY_SERVER_AS_SUB_PROCESS = True
+
+    LINKPLAY_HOST = '0.0.0.0'
+    LINKPLAY_UDP_PORT = 10900
+    LINKPLAY_TCP_PORT = 10901
+    LINKPLAY_AUTHENTICATION = 'my_link_play_server'
+    LINKPLAY_DISPLAY_HOST = ''
+
+    SSL_CERT = ''
+    SSL_KEY = ''
+
+    IS_APRILFOOLS = True
+
+    WORLD_RANK_MAX = 200
+
+    AVAILABLE_MAP = []  # list[str]
+
+    USERNAME = 'admin'
+    PASSWORD = 'admin'
+
+    SECRET_KEY = '1145141919810'
+
+    API_TOKEN = ''
+
+    DOWNLOAD_LINK_PREFIX = ''
+
+    DOWNLOAD_USE_NGINX_X_ACCEL_REDIRECT = False
+    NGINX_X_ACCEL_REDIRECT_PREFIX = '/nginx_download/'
+
+    DOWNLOAD_TIMES_LIMIT = 3000
+    DOWNLOAD_TIME_GAP_LIMIT = 1000
+
+    LOGIN_DEVICE_NUMBER_LIMIT = 1
+    ALLOW_LOGIN_SAME_DEVICE = False
+    ALLOW_BAN_MULTIDEVICE_USER_AUTO = True
+
+    ALLOW_INFO_LOG = False
+    ALLOW_WARNING_LOG = False
+
+    DEFAULT_MEMORIES = 0
+
+    UPDATE_WITH_NEW_CHARACTER_DATA = True
+
+    CHARACTER_FULL_UNLOCK = True
+    WORLD_SONG_FULL_UNLOCK = True
+    WORLD_SCENERY_FULL_UNLOCK = True
+
+    SAVE_FULL_UNLOCK = False
+
+    # ------------------------------------------
+
+    # You can change this to make another PTT mechanism.
+    BEST30_WEIGHT = 1 / 40
+    RECENT10_WEIGHT = 1 / 40
+
+    MAX_FRIEND_COUNT = 50
+
+    WORLD_MAP_FOLDER_PATH = './database/map/'
+    SONG_FILE_FOLDER_PATH = './database/songs/'
+    SONGLIST_FILE_PATH = './database/songs/songlist'
+    SQLITE_DATABASE_PATH = './database/arcaea_database.db'
+    SQLITE_DATABASE_BACKUP_FOLDER_PATH = './database/backup/'
+    DATABASE_INIT_PATH = './database/init/'
+
+    GAME_LOGIN_RATE_LIMIT = '30/5 minutes'
+    API_LOGIN_RATE_LIMIT = '10/5 minutes'
+
+
+class ConfigManager:
+
+    @staticmethod
+    def load(config) -> None:
+        for k, v in config.__dict__.items():
+            if k.startswith('__') or k.endswith('__'):
+                continue
+            if hasattr(Config, k):
+                setattr(Config, k, v)

latest version/core/constant.py

@@ -1,4 +1,6 @@
-from setting import Config
+from .config_manager import Config
+
+ARCAEA_SERVER_VERSION = 'v2.10.1'
 
 
 class Constant:
@@ -9,6 +11,8 @@ class Constant:
 
     STAMINA_RECOVER_TICK = 1800000
 
+    COURSE_STAMINA_COST = 4
+
     CORE_EXP = 250
 
     LEVEL_STEPS = {1: 0, 2: 50, 3: 100, 4: 150, 5: 200, 6: 300, 7: 450, 8: 650, 9: 900, 10: 1200, 11: 1600, 12: 2100, 13: 2700, 14: 3400, 15: 4200, 16: 5100,
@@ -19,19 +23,18 @@ class Constant:
     AYU_UNCAP_BONUS_PROGRESS = 5
     SKILL_FATALIS_WORLD_LOCKED_TIME = 3600000
 
-    MAX_FRIEND_COUNT = 50
+    MAX_FRIEND_COUNT = Config.MAX_FRIEND_COUNT
 
     MY_RANK_MAX_LOCAL_POSITION = 5
     MY_RANK_MAX_GLOBAL_POSITION = 9999
 
-    # You can change this to make another PTT mechanism.
+    BEST30_WEIGHT = Config.BEST30_WEIGHT
-    BEST30_WEIGHT = 1 / 40
+    RECENT10_WEIGHT = Config.RECENT10_WEIGHT
-    RECENT10_WEIGHT = 1 / 40
 
-    WORLD_MAP_FOLDER_PATH = './database/map/'
+    WORLD_MAP_FOLDER_PATH = Config.WORLD_MAP_FOLDER_PATH
-    SONG_FILE_FOLDER_PATH = './database/songs/'
+    SONG_FILE_FOLDER_PATH = Config.SONG_FILE_FOLDER_PATH
-    SONGLIST_FILE_PATH = './database/songs/songlist'
+    SONGLIST_FILE_PATH = Config.SONGLIST_FILE_PATH
-    SQLITE_DATABASE_PATH = './database/arcaea_database.db'
+    SQLITE_DATABASE_PATH = Config.SQLITE_DATABASE_PATH
 
     DOWNLOAD_TIMES_LIMIT = Config.DOWNLOAD_TIMES_LIMIT
     DOWNLOAD_TIME_GAP_LIMIT = Config.DOWNLOAD_TIME_GAP_LIMIT
@@ -45,8 +48,6 @@ class Constant:
     LINKPLAY_UDP_PORT = Config.LINKPLAY_UDP_PORT
     LINKPLAY_AUTHENTICATION = Config.LINKPLAY_AUTHENTICATION
 
-    COURSE_STAMINA_COST = 4
-
     # Well, I can't say a word when I see this.
     FINALE_SWITCH = [
         (0x0015F0, 0x00B032), (0x014C9A, 0x014408), (0x062585, 0x02783B),
@@ -93,3 +94,8 @@ class Constant:
         (0X5D6FC5, 0xab97ef), (0X237206D, 0xdfef2), (0XA3DEE,
                                                      0x6CB300), (0XA35687B, 0xE456CDEA)
     ]
+
+    DATABASE_MIGRATE_TABLES = ['user', 'friend', 'best_score', 'recent30', 'user_world', 'item', 'user_item', 'purchase', 'purchase_item', 'user_save',
+                               'login', 'present', 'user_present', 'present_item', 'redeem', 'user_redeem', 'redeem_item', 'api_login', 'chart', 'user_course', 'user_char', 'user_role']
+
+    UPDATE_WITH_NEW_CHARACTER_DATA = Config.UPDATE_WITH_NEW_CHARACTER_DATA

latest version/core/download.py

@@ -7,6 +7,7 @@ from flask import url_for
 
 from .constant import Constant
 from .error import NoAccess
+from .limiter import ArcLimiter
 from .user import User
 from .util import get_file_md5, md5
 
@@ -28,20 +29,69 @@ def initialize_songfile():
     del x
 
 
-@lru_cache()
+class SonglistParser:
-def get_only_3_song_ids():
+    '''songlist文件解析器'''
-    '''初始化只能下载byd相关的歌曲id'''
+
-    if not os.path.isfile(Constant.SONGLIST_FILE_PATH):
+    FILE_NAMES = ['0.aff', '1.aff', '2.aff', '3.aff',
-        return []
+                  'base.ogg', '3.ogg', 'video.mp4', 'video_audio.ogg']
-    only_3_song_ids = []
+    songs: dict = {}  # {song_id: value, ...}
-    data = []
+    # value: bit 76543210
-    with open(Constant.SONGLIST_FILE_PATH, 'r', encoding='utf-8') as f:
+    # 7: video_audio.ogg
-        data = loads(f.read())['songs']
+    # 6: video.mp4
-    for x in data:
+    # 5: 3.ogg
-        if 'remote_dl' not in x or 'remote_dl' in x and not x['remote_dl']:
+    # 4: base.ogg
-            if any(i['ratingClass'] == 3 for i in x['difficulties']):
+    # 3: 3.aff
-                only_3_song_ids.append(x['id'])
+    # 2: 2.aff
-    return only_3_song_ids
+    # 1: 1.aff
+    # 0: 0.aff
+
+    def __init__(self, path=Constant.SONGLIST_FILE_PATH) -> None:
+        self.path = path
+        self.data: list = []
+        self.parse()
+
+    @staticmethod
+    def is_available_file(song_id: str, file_name: str) -> list:
+        '''判断文件是否允许被下载'''
+        if song_id not in SonglistParser.songs:
+            # songlist没有，则只限制文件名
+            return file_name in SonglistParser.FILE_NAMES
+        rule = SonglistParser.songs[song_id]
+        for i in range(8):
+            if file_name == SonglistParser.FILE_NAMES[i] and rule & (1 << i) != 0:
+                return True
+        return False
+
+    def parse_one(self, song: dict) -> dict:
+        '''解析单个歌曲'''
+        if not 'id' in song:
+            return None
+        r = 0
+        if 'remote_dl' in song and song['remote_dl']:
+            r |= 16
+            for i in song.get('difficulties', []):
+                if i['ratingClass'] == 3 and i.get('audioOverride', False):
+                    r |= 32
+                r |= 1 << i['ratingClass']
+        else:
+            if any(i['ratingClass'] == 3 for i in song.get('difficulties', [])):
+                r |= 8
+
+        if 'additional_files' in song:
+            if 'video.mp4' in song['additional_files']:
+                r |= 64
+            if 'video_audio.ogg' in song['additional_files']:
+                r |= 128
+        return {song['id']: r}
+
+    def parse(self) -> None:
+        '''解析songlist文件'''
+        if not os.path.isfile(self.path):
+            return
+        with open(self.path, 'r', encoding='utf-8') as f:
+            self.data = loads(f.read()).get('songs', [])
+        for x in self.data:
+            self.songs.update(self.parse_one(x))
 
 
 class UserDownload:
@@ -50,8 +100,11 @@ class UserDownload:
         properties: `user` - `User`类或子类的实例
     '''
 
-    def __init__(self, c=None, user=None) -> None:
+    limiter = ArcLimiter(
-        self.c = c
+        str(Constant.DOWNLOAD_TIMES_LIMIT) + '/day', 'download')
+
+    def __init__(self, c_m=None, user=None) -> None:
+        self.c_m = c_m
         self.user = user
 
         self.song_id: str = None
@@ -60,19 +113,13 @@ class UserDownload:
         self.token: str = None
         self.token_time: int = None
 
-    def clear_user_download(self) -> None:
-        self.c.execute(
-            '''delete from user_download where user_id = :a and time <= :b''', {'a': self.user.user_id, 'b': int(time()) - 24*3600})
-
     @property
     def is_limited(self) -> bool:
         '''是否达到用户最大下载量'''
         if self.user is None:
             self.select_for_check()
-        self.c.execute(
+
-            '''select count(*) from user_download where user_id = :a''', {'a': self.user.user_id})
+        return not self.limiter.test(str(self.user.user_id))
-        y = self.c.fetchone()
-        return y is not None and y[0] > Constant.DOWNLOAD_TIMES_LIMIT
 
     @property
     def is_valid(self) -> bool:
@@ -81,19 +128,19 @@ class UserDownload:
             self.select_for_check()
         return int(time()) - self.token_time <= Constant.DOWNLOAD_TIME_GAP_LIMIT
 
-    def insert_user_download(self) -> None:
+    def download_hit(self) -> bool:
-        '''记录下载信息'''
+        '''下载次数+1，返回成功与否bool值'''
-        self.c.execute('''insert into user_download values(:a,:b,:c)''', {
+        return self.limiter.hit(str(self.user.user_id))
-                       'a': self.user.user_id, 'c': self.token, 'b': int(time())})
 
     def select_for_check(self) -> None:
         '''利用token、song_id、file_name查询其它信息'''
-        self.c.execute('''select user_id, time from download_token where song_id=? and file_name=? and token = ? limit 1;''',
+        self.c_m.execute('''select user_id, time from download_token where song_id=? and file_name=? and token = ? limit 1;''',
-                       (self.song_id, self.file_name, self.token))
+                         (self.song_id, self.file_name, self.token))
 
-        x = self.c.fetchone()
+        x = self.c_m.fetchone()
         if not x:
-            raise NoAccess('The token `%s` is not valid.' % self.token, status=403)
+            raise NoAccess('The token `%s` is not valid.' %
+                           self.token, status=403)
         self.user = User()
         self.user.user_id = x[0]
         self.token_time = x[1]
@@ -101,11 +148,11 @@ class UserDownload:
     def generate_token(self) -> None:
         self.token_time = int(time())
         self.token = md5(str(self.user.user_id) + self.song_id +
-                         self.file_name + str(self.token_time))
+                         self.file_name + str(self.token_time) + str(os.urandom(8)))
 
     def insert_download_token(self) -> None:
         '''将数据插入数据库，让这个下载链接可用'''
-        self.c.execute('''insert into download_token values(:a,:b,:c,:d,:e)''', {
+        self.c_m.execute('''insert or replace into download_token values(:a,:b,:c,:d,:e)''', {
             'a': self.user.user_id, 'b': self.song_id, 'c': self.file_name, 'd': self.token, 'e': self.token_time})
 
     @property
@@ -113,14 +160,14 @@ class UserDownload:
         '''生成下载链接'''
         if self.token is None:
             self.generate_token()
-            self.insert_download_token()
+            # self.insert_download_token() # 循环插入速度慢，改为executemany
         if Constant.DOWNLOAD_LINK_PREFIX:
             prefix = Constant.DOWNLOAD_LINK_PREFIX
             if prefix[-1] != '/':
                 prefix += '/'
-            return prefix + self.song_id + '/' + self.file_name + '?t=' + self.token
+            return f'{prefix}{self.song_id}/{self.file_name}?t={self.token}'
         else:
-            return url_for('download', file_path=self.song_id + '/' + self.file_name, t=self.token, _external=True)
+            return url_for('download', file_path=f'{self.song_id}/{self.file_name}', t=self.token, _external=True)
 
     @property
     def hash(self) -> str:
@@ -133,8 +180,8 @@ class DownloadList(UserDownload):
         properties: `user` - `User`类或子类的实例
     '''
 
-    def __init__(self, c=None, user=None) -> None:
+    def __init__(self, c_m=None, user=None) -> None:
-        super().__init__(c, user)
+        super().__init__(c_m, user)
 
         self.song_ids: list = None
         self.url_flag: bool = None
@@ -142,68 +189,98 @@ class DownloadList(UserDownload):
         self.downloads: list = []
         self.urls: dict = {}
 
-    def clear_download_token_from_song(self, song_id: str) -> None:
+    @staticmethod
-        self.c.execute('''delete from download_token where user_id=:a and song_id=:b''', {
+    def clear_all_cache():
-            'a': self.user.user_id, 'b': song_id})
+        '''清除所有歌曲文件有关缓存'''
+        get_song_file_md5.cache_clear()
+        DownloadList.get_one_song_file_names.cache_clear()
+        DownloadList.get_all_song_ids.cache_clear()
+        SonglistParser()
+
+    def clear_download_token(self) -> None:
+        '''清除过期下载链接'''
+        self.c_m.execute('''delete from download_token where time<?''',
+                         (int(time()) - Constant.DOWNLOAD_TIME_GAP_LIMIT,))
+
+    def insert_download_tokens(self) -> None:
+        '''插入所有下载链接'''
+        self.c_m.executemany('''insert or replace into download_token values(?,?,?,?,?)''', [(
+            self.user.user_id, x.song_id, x.file_name, x.token, x.token_time) for x in self.downloads])
+
+    @staticmethod
+    @lru_cache(maxsize=2048)
+    def get_one_song_file_names(song_id: str) -> list:
+        '''获取一个歌曲文件夹下的所有合法文件名，有lru缓存'''
+        r = []
+        for i in os.listdir(os.path.join(Constant.SONG_FILE_FOLDER_PATH, song_id)):
+            if os.path.isfile(os.path.join(Constant.SONG_FILE_FOLDER_PATH, song_id, i)) and SonglistParser.is_available_file(song_id, i):
+                r.append(i)
+        return r
 
     def add_one_song(self, song_id: str) -> None:
-        if self.url_flag:
-            self.clear_download_token_from_song(song_id)
-        dir_list = os.listdir(os.path.join(
-            Constant.SONG_FILE_FOLDER_PATH, song_id))
 
         re = {}
-        for i in dir_list:
+        for i in self.get_one_song_file_names(song_id):
-            if os.path.isfile(os.path.join(Constant.SONG_FILE_FOLDER_PATH, song_id, i)) and i in ['0.aff', '1.aff', '2.aff', '3.aff', 'base.ogg', '3.ogg', 'video.mp4', 'video_audio.ogg']:
+            x = UserDownload(self.c_m, self.user)
-                if song_id in get_only_3_song_ids() and i not in ['3.aff', '3.ogg']:
+            self.downloads.append(x)
-                    continue
+            x.song_id = song_id
-                x = UserDownload(self.c, self.user)
+            x.file_name = i
-                # self.downloads.append(x) # 这实际上没有用
+            if i == 'base.ogg':
-                x.song_id = song_id
+                if 'audio' not in re:
-                x.file_name = i
+                    re['audio'] = {}
-                if i == 'base.ogg':
-                    if 'audio' not in re:
-                        re['audio'] = {}
 
-                    re['audio']["checksum"] = x.hash
+                re['audio']["checksum"] = x.hash
-                    if self.url_flag:
+                if self.url_flag:
-                        re['audio']["url"] = x.url
+                    re['audio']["url"] = x.url
-                elif i == '3.ogg':
+            elif i == '3.ogg':
-                    if 'audio' not in re:
+                if 'audio' not in re:
-                        re['audio'] = {}
+                    re['audio'] = {}
 
-                    if self.url_flag:
+                if self.url_flag:
-                        re['audio']['3'] = {"checksum": x.hash, "url": x.url}
+                    re['audio']['3'] = {"checksum": x.hash, "url": x.url}
-                    else:
-                        re['audio']['3'] = {"checksum": x.hash}
-                elif i == 'video.mp4' or i == 'video_audio.ogg':
-                    if 'additional_files' not in re:
-                        re['additional_files'] = []
-
-                    if self.url_flag:
-                        re['additional_files'].append(
-                            {"checksum": x.hash, "url": x.url, 'file_name': i})
-                    else:
-                        re['additional_files'].append(
-                            {"checksum": x.hash, 'file_name': i})
                 else:
-                    if 'chart' not in re:
+                    re['audio']['3'] = {"checksum": x.hash}
-                        re['chart'] = {}
+            elif i == 'video.mp4' or i == 'video_audio.ogg':
+                if 'additional_files' not in re:
+                    re['additional_files'] = []
 
-                    if self.url_flag:
+                if self.url_flag:
-                        re['chart'][i[0]] = {"checksum": x.hash, "url": x.url}
+                    re['additional_files'].append(
-                    else:
+                        {"checksum": x.hash, "url": x.url, 'file_name': i})
-                        re['chart'][i[0]] = {"checksum": x.hash}
+                else:
+                    re['additional_files'].append(
+                        {"checksum": x.hash, 'file_name': i})
+            else:
+                if 'chart' not in re:
+                    re['chart'] = {}
+
+                if self.url_flag:
+                    re['chart'][i[0]] = {"checksum": x.hash, "url": x.url}
+                else:
+                    re['chart'][i[0]] = {"checksum": x.hash}
 
         self.urls.update({song_id: re})
 
+    @staticmethod
+    @lru_cache()
+    def get_all_song_ids() -> list:
+        '''获取全歌曲文件夹列表，有lru缓存'''
+        return list(filter(lambda x: os.path.isdir(os.path.join(Constant.SONG_FILE_FOLDER_PATH, x)), os.listdir(Constant.SONG_FILE_FOLDER_PATH)))
+
     def add_songs(self, song_ids: list = None) -> None:
         '''添加一个或多个歌曲到下载列表，若`song_ids`为空，则添加所有歌曲'''
         if song_ids is not None:
             self.song_ids = song_ids
 
-        x = self.song_ids if self.song_ids else os.listdir(
+        if not self.song_ids:
-            Constant.SONG_FILE_FOLDER_PATH)
+            self.song_ids = self.get_all_song_ids()
-        for i in x:
+            for i in self.song_ids:
-            if os.path.isdir(os.path.join(Constant.SONG_FILE_FOLDER_PATH, i)):
                 self.add_one_song(i)
+        else:
+            for i in self.song_ids:
+                if os.path.isdir(os.path.join(Constant.SONG_FILE_FOLDER_PATH, i)):
+                    self.add_one_song(i)
+
+        if self.url_flag:
+            self.clear_download_token()
+            self.insert_download_tokens()

latest version/core/error.py

@@ -1,91 +1,111 @@
 class ArcError(Exception):
     def __init__(self, message=None, error_code=108, api_error_code=-999, extra_data=None, status=200) -> None:
-        self.message = message
+        self.message: str = message
-        self.error_code = error_code
+        self.error_code: int = error_code
-        self.api_error_code = api_error_code
+        self.api_error_code: int = api_error_code
         self.extra_data = extra_data
-        self.status = status
+        self.status: int = status
 
     def __str__(self) -> str:
         return repr(self.message)
 
 
 class InputError(ArcError):
-    # 输入类型错误
+    '''输入类型错误'''
+
     def __init__(self, message=None, error_code=108, api_error_code=-100, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class DataExist(ArcError):
-    # 数据存在
+    '''数据存在'''
     pass
 
 
 class NoData(ArcError):
-    # 数据不存在
+    '''数据不存在'''
+
     def __init__(self, message=None, error_code=108, api_error_code=-2, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class PostError(ArcError):
-    # 缺少输入
+    '''缺少输入'''
+
     def __init__(self, message=None, error_code=108, api_error_code=-100, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class UserBan(ArcError):
-    # 用户封禁
+    '''用户封禁'''
+
     def __init__(self, message=None, error_code=121, api_error_code=-202, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class ItemNotEnough(ArcError):
-    # 物品数量不足
+    '''物品数量不足'''
+
     def __init__(self, message=None, error_code=-6, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class ItemUnavailable(ArcError):
-    # 物品不可用
+    '''物品不可用'''
+
     def __init__(self, message=None, error_code=-6, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class RedeemUnavailable(ArcError):
-    # 兑换码不可用
+    '''兑换码不可用'''
+
     def __init__(self, message=None, error_code=505, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class MapLocked(ArcError):
-    # 地图锁定
+    '''地图锁定'''
+
     def __init__(self, message=None, error_code=108, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class StaminaNotEnough(ArcError):
-    # 体力不足
+    '''体力不足'''
+
     def __init__(self, message=None, error_code=107, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class TicketNotEnough(ArcError):
-    # 记忆源点不足
+    '''记忆源点不足'''
+
     def __init__(self, message=None, error_code=-6, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class FriendError(ArcError):
-    # 好友系统出错
+    '''好友系统出错'''
+
     def __init__(self, message=None, error_code=108, api_error_code=-999, extra_data=None, status=200) -> None:
         super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class NoAccess(ArcError):
-    # 无权限
+    '''无权限'''
-    pass
+
+    def __init__(self, message=None, error_code=108, api_error_code=-999, extra_data=None, status=403) -> None:
+        super().__init__(message, error_code, api_error_code, extra_data, status)
 
 
 class Timeout(ArcError):
-    # 超时
+    '''超时'''
     pass
+
+
+class RateLimit(ArcError):
+    '''频率达到限制'''
+
+    def __init__(self, message=None, error_code=108, api_error_code=-999, extra_data=None, status=429) -> None:
+        super().__init__(message, error_code, api_error_code, extra_data, status)

latest version/core/init.py

@@ -0,0 +1,248 @@
+import os
+import sys
+from importlib import import_module
+from json import load
+from shutil import copy, copy2
+from time import time
+
+from core.config_manager import Config
+from core.constant import ARCAEA_SERVER_VERSION
+from core.course import Course
+from core.download import SonglistParser
+from core.purchase import Purchase
+from core.sql import Connect, DatabaseMigrator, MemoryDatabase
+from core.user import UserRegister
+from core.util import try_rename
+
+
+class DatabaseInit:
+    def __init__(self, db_path: str = Config.SQLITE_DATABASE_PATH, init_folder_path: str = Config.DATABASE_INIT_PATH) -> None:
+        self.db_path = db_path
+        self.init_folder_path = init_folder_path
+        self.c = None
+        self.init_data = None
+
+    @property
+    def sql_path(self) -> str:
+        return os.path.join(self.init_folder_path, 'tables.sql')
+
+    @property
+    def course_path(self) -> str:
+        return os.path.join(self.init_folder_path, 'courses.json')
+
+    @property
+    def pack_path(self) -> str:
+        return os.path.join(self.init_folder_path, 'packs.json')
+
+    @property
+    def single_path(self) -> str:
+        return os.path.join(self.init_folder_path, 'singles.json')
+
+    def table_init(self) -> None:
+        '''初始化数据库结构'''
+        with open(self.sql_path, 'r') as f:
+            self.c.executescript(f.read())
+        self.c.execute('''insert into config values("version", :a);''', {
+            'a': ARCAEA_SERVER_VERSION})
+
+    def character_init(self) -> None:
+        '''初始化搭档信息'''
+        for i in range(0, len(self.init_data.char)):
+            skill_requires_uncap = 1 if i == 2 else 0
+            if i in [0, 1, 2, 4, 13, 26, 27, 28, 29, 36, 21, 42, 43, 11, 12, 19, 5]:
+                max_level = 30
+                uncapable = 1
+            else:
+                max_level = 20
+                uncapable = 0
+            sql = '''insert into character values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)'''
+            self.c.execute(sql, (i, self.init_data.char[i], max_level, self.init_data.frag1[i], self.init_data.prog1[i], self.init_data.overdrive1[i], self.init_data.frag20[i], self.init_data.prog20[i], self.init_data.overdrive20[i],
+                                 self.init_data.frag30[i], self.init_data.prog30[i], self.init_data.overdrive30[i], self.init_data.skill_id[i], self.init_data.skill_unlock_level[i], skill_requires_uncap, self.init_data.skill_id_uncap[i], self.init_data.char_type[i], uncapable))
+
+        self.c.execute('''insert into character values(?,?,20,?,?,?,?,?,?,?,?,?,?,?,?,?,?,0)''', (
+            99, 'shirahime', 38, 33, 28, 66, 58, 50, 66, 58, 50, 'frags_preferred_song', 0, 0, '', 0))
+
+        for i in self.init_data.char_core:
+            self.c.executemany('''insert into char_item values(?,?,'core',?)''', [
+                (i, j['core_id'], j['amount']) for j in self.init_data.char_core[i]])
+
+    def insert_purchase_item(self, purchases: list):
+        '''处理singles和packs'''
+        for i in purchases:
+            x = Purchase(self.c).from_dict(i)
+            x.insert_all()
+
+    def item_init(self) -> None:
+        '''初始化物品信息'''
+        self.c.executemany('''insert into item values(?,"core",1)''',
+                           [(i,) for i in self.init_data.cores])
+        self.c.executemany('''insert into item values(?,"world_song",1)''', [
+            (i,) for i in self.init_data.world_songs])
+        self.c.executemany('''insert into item values(?,"world_unlock",1)''', [
+            (i,) for i in self.init_data.world_unlocks])
+        self.c.executemany('''insert into item values(?,"course_banner",1)''', [
+            (i,) for i in self.init_data.course_banners])
+
+        self.c.execute('''insert into item values(?,?,?)''',
+                       ('fragment', 'fragment', 1))
+        self.c.execute('''insert into item values(?,?,?)''',
+                       ('memory', 'memory', 1))
+        self.c.execute('''insert into item values(?,?,?)''',
+                       ('anni5tix', 'anni5tix', 1))
+
+        with open(self.pack_path, 'r') as f:
+            self.insert_purchase_item(load(f))
+
+        with open(self.single_path, 'r') as f:
+            self.insert_purchase_item(load(f))
+
+        self.c.execute(
+            '''select exists(select * from item where item_id='epilogue')''')
+        if self.c.fetchone() == (0,):
+            self.c.execute(
+                '''insert into item values('epilogue','pack',1)''')
+
+    def course_init(self) -> None:
+        '''初始化课题信息'''
+        courses = []
+        with open(self.course_path, 'r', encoding='utf-8') as f:
+            courses = load(f)
+        for i in courses:
+            x = Course(self.c).from_dict(i)
+            x.insert_all()
+
+    def role_power_init(self) -> None:
+        '''初始化power和role'''
+        self.c.executemany('''insert into role values(?,?)''', [(
+            self.init_data.role[i], self.init_data.role_caption[i]) for i in range(len(self.init_data.role))])
+
+        self.c.executemany('''insert into power values(?,?)''', [(
+            self.init_data.power[i], self.init_data.power_caption[i]) for i in range(len(self.init_data.power))])
+
+        for i in self.init_data.role_power:
+            self.c.executemany('''insert into role_power values(?,?)''', [
+                               (i, j) for j in self.init_data.role_power[i]])
+
+    def admin_init(self) -> None:
+        '''初始化测试账号'''
+        x = UserRegister(self.c)
+        x.user_code = '123456789'
+        x.user_id = 2000000
+        x.name = 'admin'
+        x.email = 'admin@admin.com'
+        now = int(time() * 1000)
+
+        x._insert_user_char()
+
+        self.c.execute('''insert into user(user_id, name, password, join_date, user_code, rating_ptt, 
+        character_id, is_skill_sealed, is_char_uncapped, is_char_uncapped_override, is_hide_rating, favorite_character, max_stamina_notification_enabled, current_map, ticket, prog_boost, email)
+        values(:user_id, :name, :password, :join_date, :user_code, 0, 0, 0, 0, 0, 0, -1, 0, '', :memories, 0, :email)
+        ''', {'user_code': x.user_code, 'user_id': x.user_id, 'join_date': now, 'name': x.name, 'password': '41e5653fc7aeb894026d6bb7b2db7f65902b454945fa8fd65a6327047b5277fb', 'memories': 114514, 'email': x.email})
+        self.c.execute('''insert into recent30(user_id) values(:user_id)''', {
+                       'user_id': x.user_id})
+        self.c.execute(
+            '''insert into user_role values(?, "admin")''', (x.user_id,))
+
+    def init(self) -> None:
+        sys.path.append(os.path.join(sys.path[0], self.init_folder_path))
+        self.init_data = import_module('arc_data').InitData
+
+        with Connect(self.db_path) as c:
+            self.c = c
+            self.table_init()
+            self.character_init()
+            self.item_init()
+            self.course_init()
+            self.role_power_init()
+            self.admin_init()
+
+
+class FileChecker:
+
+    def __init__(self, app=None):
+        self.app = app
+
+    def check_file(self, file_path: str) -> bool:
+        f = os.path.isfile(file_path)
+        if not f:
+            self.app.logger.warning('File `%s` is missing.' % file_path)
+        return f
+
+    def check_folder(self, folder_path: str) -> bool:
+        f = os.path.isdir(folder_path)
+        if not f:
+            self.app.logger.warning('Folder `%s` is missing.' % folder_path)
+        return f
+
+    def check_update_database(self) -> bool:
+        if not self.check_file(Config.SQLITE_DATABASE_PATH):
+            # 新建数据库
+            try:
+                self.app.logger.info(
+                    'Try to new the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+                DatabaseInit().init()
+                self.app.logger.info(
+                    'Success to new the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+            except:
+                self.app.logger.warning(
+                    'Fail to new the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+                return False
+        else:
+            # 检查更新
+            with Connect() as c:
+                try:
+                    c.execute('''select value from config where id="version"''')
+                    x = c.fetchone()
+                except:
+                    x = None
+            # 数据库自动更新，不强求
+            if not x or x[0] != ARCAEA_SERVER_VERSION:
+                self.app.logger.warning(
+                    'Maybe the file `%s` is an old version.' % Config.SQLITE_DATABASE_PATH)
+                try:
+                    self.app.logger.info(
+                        'Try to update the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+
+                    if not os.path.isdir(Config.SQLITE_DATABASE_BACKUP_FOLDER_PATH):
+                        os.makedirs(Config.SQLITE_DATABASE_BACKUP_FOLDER_PATH)
+
+                    backup_path = try_rename(Config.SQLITE_DATABASE_PATH, os.path.join(
+                        Config.SQLITE_DATABASE_BACKUP_FOLDER_PATH, os.path.split(Config.SQLITE_DATABASE_PATH)[-1] + '.bak'))
+
+                    try:
+                        copy2(backup_path, Config.SQLITE_DATABASE_PATH)
+                    except:
+                        copy(backup_path, Config.SQLITE_DATABASE_PATH)
+
+                    temp_path = os.path.join(
+                        *os.path.split(Config.SQLITE_DATABASE_PATH)[:-1], 'old_arcaea_database.db')
+                    if os.path.isfile(temp_path):
+                        os.remove(temp_path)
+
+                    try_rename(Config.SQLITE_DATABASE_PATH, temp_path)
+
+                    DatabaseInit().init()
+                    self.update_database(temp_path)
+
+                    self.app.logger.info(
+                        'Success to update the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+
+                except ValueError:
+                    self.app.logger.warning(
+                        'Fail to update the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+
+        return True
+
+    @staticmethod
+    def update_database(old_path: str, new_path: str = Config.SQLITE_DATABASE_PATH) -> None:
+        '''更新数据库，并删除旧文件'''
+        if os.path.isfile(old_path) and os.path.isfile(new_path):
+            DatabaseMigrator(old_path, new_path).update_database()
+            os.remove(old_path)
+
+    def check_before_run(self) -> bool:
+        '''运行前检查，返回布尔值'''
+        # TODO: try
+        MemoryDatabase()  # 初始化内存数据库
+        SonglistParser()  # 解析songlist
+        return self.check_folder(Config.SONG_FILE_FOLDER_PATH) & self.check_update_database()

latest version/core/item.py

@@ -1,5 +1,5 @@
+from .config_manager import Config
 from .error import InputError, ItemNotEnough, ItemUnavailable, NoData
-from setting import Config
 
 
 class Item:

latest version/core/limiter.py

@@ -0,0 +1,31 @@
+from limits import parse_many, strategies
+from limits.storage import storage_from_string
+
+
+class ArcLimiter:
+    storage = storage_from_string("memory://")
+    strategy = strategies.FixedWindowRateLimiter(storage)
+
+    def __init__(self, limit_str: str = None, namespace: str = None):
+        self._limits: list = None
+        self.limits = limit_str
+        self.namespace = namespace
+
+    @property
+    def limits(self) -> list:
+        return self._limits
+
+    @limits.setter
+    def limits(self, value: str):
+        if value is None:
+            return
+        self._limits = parse_many(value)
+
+    def hit(self, key: str, cost: int = 1) -> bool:
+        flag = True
+        for limit in self.limits:
+            flag &= self.strategy.hit(limit, self.namespace, key, cost)
+        return flag
+
+    def test(self, key: str) -> bool:
+        return all(self.strategy.test(limit, self.namespace, key) for limit in self.limits)

latest version/core/purchase.py

@@ -53,6 +53,36 @@ class Purchase:
                 r['discount_reason'] = self.discount_reason
         return r
 
+    def from_dict(self, d: dict) -> 'Purchase':
+        self.purchase_name = d['name']
+        self.price = d['price']
+        self.orig_price = d['orig_price']
+        self.discount_from = d.get('discount_from', -1)
+        self.discount_to = d.get('discount_to', -1)
+        self.discount_reason = d.get('discount_reason', '')
+        for i in d.get('items', []):
+            self.items.append(ItemFactory.from_dict(i))
+
+        return self
+
+    def insert_all(self) -> None:
+        '''向数据库插入，包括item表和purchase_item表'''
+        self.c.execute('''insert into purchase values(?,?,?,?,?,?)''',
+                       (self.purchase_name, self.price, self.orig_price, self.discount_from, self.discount_to, self.discount_reason))
+        self.insert_items()
+
+    def insert_items(self) -> None:
+        '''向数据库插入物品，注意已存在的物品不会变更'''
+        for i in self.items:
+            self.c.execute(
+                '''select exists(select * from item where item_id=?)''', (i.item_id,))
+            if self.c.fetchone() == (0,):
+                self.c.execute('''insert into item values(?,?,?)''',
+                               (i.item_id, i.item_type, i.is_available))
+
+            self.c.execute('''insert into purchase_item values(?,?,?,?)''',
+                           (self.purchase_name, i.item_id, i.item_type, i.amount))
+
     def select(self, purchase_name: str = None) -> 'Purchase':
         '''
             用purchase_name查询信息

latest version/core/rank.py

@@ -1,6 +1,7 @@
 from .constant import Constant
 from .score import UserScore
 from .song import Chart
+from .sql import Query, Sql
 from .user import UserInfo
 
 
@@ -26,24 +27,29 @@ class RankList:
             得到top分数表
         '''
         if self.limit >= 0:
-            self.c.execute('''select user_id from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC limit :limit''', {
+            self.c.execute('''select * from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC limit :limit''', {
                 'song_id': self.song.song_id, 'difficulty': self.song.difficulty, 'limit': self.limit})
         else:
-            self.c.execute('''select user_id from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC''', {
+            self.c.execute('''select * from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC''', {
                 'song_id': self.song.song_id, 'difficulty': self.song.difficulty})
 
         x = self.c.fetchall()
         if not x:
             return None
 
+        user_info_list = Sql(self.c).select('user', ['user_id', 'name', 'character_id', 'is_skill_sealed', 'is_char_uncapped',
+                                                     'is_char_uncapped_override', 'favorite_character'], Query().from_args({'user_id': [i[0] for i in x]}))
+        user_info_dict = {i[0]: i[1:] for i in user_info_list}
+
         rank = 0
         self.list = []
         for i in x:
             rank += 1
-            y = UserScore(self.c, UserInfo(self.c, i[0]))
+            y = UserScore(self.c, UserInfo(self.c, i[0])).from_list(i)
             y.song = self.song
-            y.select_score()
             y.rank = rank
+            y.user.from_list_about_character(user_info_dict[i[0]])
+
             self.list.append(y)
 
     def select_friend(self, user=None, limit=Constant.MAX_FRIEND_COUNT) -> None:
@@ -54,20 +60,26 @@ class RankList:
         if user:
             self.user = user
 
-        self.c.execute('''select user_id from best_score where user_id in (select :user_id union select user_id_other from friend where user_id_me = :user_id) and song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC limit :limit''', {
+        user_ids = [self.user.user_id] + [x[0] for x in self.user.friend_ids]
-            'user_id': self.user.user_id, 'song_id': self.song.song_id, 'difficulty': self.song.difficulty, 'limit': self.limit})
+
+        self.c.execute(f'''select * from best_score where user_id in ({','.join(['?'] * len(user_ids))}) and song_id = ? and difficulty = ? order by score DESC, time_played DESC limit ?''', user_ids + [
+                       self.song.song_id, self.song.difficulty, self.limit])
         x = self.c.fetchall()
         if not x:
             return None
 
+        user_info_list = Sql(self.c).select('user', ['user_id', 'name', 'character_id', 'is_skill_sealed', 'is_char_uncapped',
+                                                     'is_char_uncapped_override', 'favorite_character'], Query().from_args({'user_id': [i[0] for i in x]}))
+        user_info_dict = {i[0]: i[1:] for i in user_info_list}
         rank = 0
         self.list = []
         for i in x:
             rank += 1
-            y = UserScore(self.c, UserInfo(self.c, i[0]))
+            y = UserScore(self.c, UserInfo(self.c, i[0])).from_list(i)
             y.song = self.song
-            y.select_score()
             y.rank = rank
+            y.user.from_list_about_character(user_info_dict[i[0]])
+
             self.list.append(y)
 
     @staticmethod
@@ -118,19 +130,25 @@ class RankList:
 
         sql_limit, sql_offset, need_myself = self.get_my_rank_parameter(
             my_rank, int(self.c.fetchone()[0]), self.limit)
-        self.c.execute('''select user_id from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC limit :limit offset :offset''', {
+        self.c.execute('''select * from best_score where song_id = :song_id and difficulty = :difficulty order by score DESC, time_played DESC limit :limit offset :offset''', {
             'song_id': self.song.song_id, 'difficulty': self.song.difficulty, 'limit': sql_limit, 'offset': sql_offset})
         x = self.c.fetchall()
+
         if x:
+            user_info_list = Sql(self.c).select('user', ['user_id', 'name', 'character_id', 'is_skill_sealed', 'is_char_uncapped',
+                                                         'is_char_uncapped_override', 'favorite_character'], Query().from_args({'user_id': [i[0] for i in x]}))
+            user_info_dict = {i[0]: i[1:] for i in user_info_list}
             rank = sql_offset if sql_offset > 0 else 0
             self.list = []
             for i in x:
                 rank += 1
-                y = UserScore(self.c, UserInfo(self.c, i[0]))
+                y = UserScore(self.c, UserInfo(self.c, i[0])).from_list(i)
                 y.song = self.song
-                y.select_score()
                 y.rank = rank
+                y.user.from_list_about_character(user_info_dict[i[0]])
+
                 self.list.append(y)
+
             if need_myself:
                 y = UserScore(self.c, UserInfo(self.c, self.user.user_id))
                 y.song = self.song

latest version/core/save.py

@@ -1,10 +1,8 @@
 import json
 from time import time
 
-from setting import Config
+from .config_manager import Config
-
+from .constant import Constant
-from core.constant import Constant
-
 from .error import InputError
 from .util import md5
 
@@ -101,9 +99,7 @@ class SaveData:
             parameter: `user` - `User`类或子类的实例
         '''
         self.createdAt = int(time() * 1000)
-        self.c.execute('''delete from user_save where user_id=:a''', {
+        self.c.execute('''insert or replace into user_save values(:a,:b,:c,:d,:e,:f,:g,:h,:i,:j)''', {
-                       'a': user.user_id})
-        self.c.execute('''insert into user_save values(:a,:b,:c,:d,:e,:f,:g,:h,:i,:j)''', {
             'a': user.user_id, 'b': json.dumps({'': self.scores_data}), 'c': json.dumps({'': self.clearlamps_data}), 'd': json.dumps({'': self.clearedsongs_data}), 'e': json.dumps({'': self.unlocklist_data}), 'f': json.dumps({'val': self.installid_data}), 'g': json.dumps({'val': self.devicemodelname_data}), 'h': json.dumps({'': self.story_data}), 'i': self.createdAt, 'j': self.finalestate_data})
 
     def set_value(self, key: str, value: str, checksum: str) -> None:

latest version/core/score.py

@@ -155,9 +155,10 @@ class UserScore(Score):
         super().__init__()
         self.c = c
         self.user = user
-        self.rank = None
+        self.rank = None  # 成绩排名，给Ranklist用的
 
     def select_score(self) -> None:
+        '''查询成绩以及用户搭档信息，单次查询可用，不要集体循环查询'''
         self.c.execute('''select * from best_score where user_id = :a and song_id = :b and difficulty = :c''',
                        {'a': self.user.user_id, 'b': self.song.song_id, 'c': self.song.difficulty})
         x = self.c.fetchone()
@@ -258,6 +259,12 @@ class UserPlay(UserScore):
 
     def get_play_state(self) -> None:
         '''检查token，当然这里不管有没有，是用来判断世界模式和课题模式的'''
+        if self.song_token == '1145141919810':
+            # 硬编码检查，绕过数据库
+            self.is_world_mode = False
+            self.course_play_state = -1
+            return None
+
         self.c.execute(
             '''select * from songplay_token where token=:a ''', {'a': self.song_token})
         x = self.c.fetchone()
@@ -265,7 +272,8 @@ class UserPlay(UserScore):
             self.is_world_mode = False
             self.course_play_state = -1
             return None
-        self.song.set_chart(x[2], x[3])
+            # raise NoData('No token data.')
+        # self.song.set_chart(x[2], x[3])
         if x[4]:
             self.course_play = CoursePlay(self.c, self.user, self)
             self.course_play.course_id = x[4]
@@ -375,12 +383,12 @@ class UserPlay(UserScore):
                 r30_id = 29
             elif self.song.song_id_difficulty in songs and n == 10:
                 i = 29
-                while self.ptt.s30[i] == self.song.song_id_difficulty:
+                while self.ptt.s30[i] == self.song.song_id_difficulty and i > 0:
                     i -= 1
                 r30_id = i
             elif self.song.song_id_difficulty not in songs and n == 9:
                 i = 29
-                while self.ptt.s30.count(self.ptt.s30[-1]) == 1:
+                while self.ptt.s30.count(self.ptt.s30[i]) == 1 and i > 0:
                     i -= 1
                 r30_id = i
             else:
@@ -487,6 +495,9 @@ class Potential:
         self.c.execute(
             '''select * from recent30 where user_id = :a''', {'a': self.user.user_id})
         x = self.c.fetchone()
+        if not x:
+            raise NoData(
+                f'No recent30 data for user `{self.user.user_id}`', api_error_code=-3)
         self.r30 = []
         self.s30 = []
         if not x:
@@ -521,11 +532,15 @@ class Potential:
     def recent_30_to_dict_list(self) -> list:
         if self.r30 is None:
             self.select_recent_30()
-        return [{
+        r = []
-            'song_id': self.s30[i][:-1],
+        for x, y in zip(self.s30, self.r30):
-            'difficulty': int(self.s30[i][-1]),
+            if x:
-            'rating': self.r30[i]
+                r.append({
-        } for i in range(len(self.r30))]
+                    'song_id': x[:-1],
+                    'difficulty': int(x[-1]),
+                    'rating': y
+                })
+        return r
 
     def recent_30_update(self, pop_index: int, rating: float, song_id_difficulty: str) -> None:
         self.r30.pop(pop_index)
@@ -580,4 +595,5 @@ class UserScoreList:
         for score in self.scores:
             self.c.execute(
                 '''select name from chart where song_id = ?''', (score.song.song_id,))
-            score.song.song_name = self.c.fetchone()[0]
+            x = self.c.fetchone()
+            score.song.song_name = x[0] if x else ''

latest version/core/sql.py

@@ -1,5 +1,6 @@
 import sqlite3
 import traceback
+from atexit import register
 
 from flask import current_app
 
@@ -10,16 +11,21 @@ from .error import ArcError, InputError
 class Connect:
     # 数据库连接类，上下文管理
 
-    def __init__(self, file_path=Constant.SQLITE_DATABASE_PATH):
+    def __init__(self, file_path: str = Constant.SQLITE_DATABASE_PATH, in_memory: bool = False):
         """
             数据库连接，默认连接arcaea_database.db\ 
             接受：文件路径\ 
             返回：sqlite3连接操作对象
         """
         self.file_path = file_path
+        self.in_memory: bool = in_memory
 
-    def __enter__(self):
+    def __enter__(self) -> sqlite3.Cursor:
-        self.conn = sqlite3.connect(self.file_path, timeout=10)
+        if self.in_memory:
+            self.conn = sqlite3.connect(
+                'file:arc_tmp?mode=memory&cache=shared', uri=True, timeout=10)
+        else:
+            self.conn = sqlite3.connect(self.file_path, timeout=10)
         self.c = self.conn.cursor()
         return self.c
 
@@ -29,15 +35,13 @@ class Connect:
             if issubclass(exc_type, ArcError):
                 flag = False
             else:
-                if self.conn:
+                self.conn.rollback()
-                    self.conn.rollback()
 
                 current_app.logger.error(
                     traceback.format_exception(exc_type, exc_val, exc_tb))
 
-        if self.conn:
+        self.conn.commit()
-            self.conn.commit()
+        self.conn.close()
-            self.conn.close()
 
         return flag
 
@@ -46,14 +50,17 @@ class Query:
     '''查询参数类'''
 
     def __init__(self, query_able: list = None, quzzy_query_able: list = None, sort_able: list = None) -> None:
-        self.query_able: list = query_able
+        self.query_able: list = query_able  # None表示不限制
-        self.quzzy_query_able: list = quzzy_query_able
+        self.quzzy_query_able: list = quzzy_query_able  # None表示不限制
         self.sort_able: list = sort_able
 
         self.__limit: int = -1
         self.__offset: int = 0
-        self.__query: dict = {}  # {'name': 'admin'}
+
+        # {'name': 'admin'} or {'name': ['admin', 'user']}
+        self.__query: dict = {}
         self.__fuzzy_query: dict = {}  # {'name': 'dmi'}
+
         # [{'column': 'user_id', 'order': 'ASC'}, ...]
         self.__sort: list = []
 
@@ -143,11 +150,15 @@ class Query:
         self.fuzzy_query = fuzzy_query
         self.sort = sort
 
-    def from_data(self, d: dict) -> 'Query':
+    def from_dict(self, d: dict) -> 'Query':
         self.set_value(d.get('limit', -1), d.get('offset', 0),
                        d.get('query', {}), d.get('fuzzy_query', {}), d.get('sort', []))
         return self
 
+    def from_args(self, query: dict, limit: int = -1, offset: int = 0, sort: list = [], fuzzy_query: dict = {}) -> 'Query':
+        self.set_value(limit, offset, query, fuzzy_query, sort)
+        return self
+
 
 class Sql:
     '''
@@ -157,63 +168,242 @@ class Sql:
     def __init__(self, c=None) -> None:
         self.c = c
 
-    def select(self, table_name: str, target_column: 'list' = [], query: 'Query' = None) -> list:
+    @staticmethod
-        '''单表内行查询单句sql语句，返回fetchall数据'''
+    def get_select_sql(table_name: str, target_column: list = [], query: 'Query' = None):
-
+        '''拼接单表内行查询单句sql语句，返回语句和参数列表'''
-        sql = 'select '
         sql_list = []
-        if len(target_column) >= 2:
+        if not target_column:
-            sql += target_column[0]
+            sql = f'select * from {table_name}'
-            for i in range(1, len(target_column)):
-                sql += ',' + target_column[i]
-            sql += ' from ' + table_name
-        elif len(target_column) == 1:
-            sql += target_column[0] + ' from ' + table_name
         else:
-            sql += '* from ' + table_name
+            sql = f"select {', '.join(target_column)} from {table_name}"
 
         if query is None:
-            self.c.execute(sql)
+            return sql, sql_list
-            return self.c.fetchall()
 
         where_key = []
-        where_like_key = []
+        for k, v in query.query.items():
-        for i in query.query:
+            if isinstance(v, list):
-            where_key.append(i)
+                where_key.append(f"{k} in ({','.join(['?'] * len(v))})")
-            sql_list.append(query.query[i])
+                sql_list.extend(v)
-
-        for i in query.fuzzy_query:
-            where_like_key.append(i)
-            sql_list.append('%' + query.fuzzy_query[i] + '%')
-
-        if where_key or where_like_key:
-            sql += ' where '
-            if where_key:
-                sql += where_key[0] + '=?'
-                if len(where_key) >= 2:
-                    for i in range(1, len(where_key)):
-                        sql += ' and ' + where_key[i] + '=?'
-                if where_like_key:
-                    for i in where_like_key:
-                        sql += ' and ' + i + ' like ?'
             else:
-                sql += where_like_key[0] + ' like ?'
+                where_key.append(f'{k}=?')
+                sql_list.append(v)
 
-                if len(where_like_key) >= 2:
+        for k, v in query.fuzzy_query.items():
-                    for i in range(1, len(where_key)):
+            where_key.append(f'{k} like ?')
-                        sql += ' and ' + where_key[i] + ' like ?'
+            sql_list.append(f'%{v}%')
+
+        if where_key:
+            sql += ' where '
+            sql += ' and '.join(where_key)
 
         if query.sort:
             sql += ' order by ' + \
-                query.sort[0]['column'] + ' ' + query.sort[0]['order']
+                ', '.join([x['column'] + ' ' + x['order'] for x in query.sort])
-            for i in range(1, len(query.sort)):
-                sql += ', ' + query.sort[i]['column'] + \
-                    ' ' + query.sort[i]['order']
 
         if query.limit >= 0:
             sql += ' limit ? offset ?'
             sql_list.append(query.limit)
             sql_list.append(query.offset)
 
+        return sql, sql_list
+
+    @staticmethod
+    def get_insert_sql(table_name: str, key: list = [], value_len: int = None, insert_type: str = None) -> str:
+        '''拼接insert语句，请注意只返回sql语句，insert_type为replace或ignore'''
+        insert_type = 'replace' if insert_type in [
+            'replace', 'R', 'r', 'REPLACE'] else 'ignore'
+        return ('insert into ' if insert_type is None else 'insert or ' + insert_type + ' into ') + table_name + ('(' + ','.join(key) + ')' if key else '') + ' values(' + ','.join(['?'] * (len(key) if value_len is None else value_len)) + ')'
+
+    @staticmethod
+    def get_update_sql(table_name: str, d: dict = {}, query: 'Query' = None) -> str:
+        if not d:
+            return None
+        sql_list = []
+        sql = f"update {table_name} set {','.join([f'{k}=?' for k in d.keys()])}"
+        sql_list.extend(d.values())
+
+        if query is None:
+            return sql, sql_list
+
+        where_key = []
+        for k, v in query.query.items():
+            if isinstance(v, list):
+                where_key.append(f"{k} in ({','.join(['?'] * len(v))})")
+                sql_list.extend(v)
+            else:
+                where_key.append(f'{k}=?')
+                sql_list.append(v)
+
+        for k, v in query.fuzzy_query.items():
+            where_key.append(f'{k} like ?')
+            sql_list.append(f'%{v}%')
+
+        if where_key:
+            sql += ' where '
+            sql += ' and '.join(where_key)
+
+        return sql, sql_list
+
+    @staticmethod
+    def get_delete_sql(table_name: str, query: 'Query' = None):
+        '''拼接删除语句，query中只有query和fuzzy_query会被处理'''
+        sql = f'delete from {table_name}'
+
+        if query is None:
+            return sql, []
+
+        sql_list = []
+        where_key = []
+        for k, v in query.query.items():
+            if isinstance(v, list):
+                where_key.append(f"{k} in ({','.join(['?'] * len(v))})")
+                sql_list.extend(v)
+            else:
+                where_key.append(f'{k}=?')
+                sql_list.append(v)
+
+        for k, v in query.fuzzy_query.items():
+            where_key.append(f'{k} like ?')
+            sql_list.append(f'%{v}%')
+
+        if where_key:
+            sql += ' where '
+            sql += ' and '.join(where_key)
+
+        return sql, sql_list
+
+    def select(self, table_name: str, target_column: list = [], query: 'Query' = None) -> list:
+        '''单表内行select单句sql语句，返回fetchall数据'''
+        sql, sql_list = self.get_select_sql(table_name, target_column, query)
         self.c.execute(sql, sql_list)
         return self.c.fetchall()
+
+    def select_exists(self, table_name: str, target_column: list = [], query: 'Query' = None) -> bool:
+        '''单表内行select exists单句sql语句，返回bool值'''
+        sql, sql_list = self.get_select_sql(table_name, target_column, query)
+        self.c.execute('select exists(' + sql + ')', sql_list)
+        return self.c.fetchone() == (1,)
+
+    def insert(self, table_name: str, key: list, value: tuple, insert_type: str = None) -> None:
+        '''单行插入或覆盖插入，key传[]则为全部列，insert_type为replace或ignore'''
+        self.c.execute(self.get_insert_sql(
+            table_name, key, len(value), insert_type), value)
+
+    def insert_many(self, table_name: str, key: list, value_list: list, insert_type: str = None) -> None:
+        '''多行插入或覆盖插入，key传[]则为全部列，insert_type为replace或ignore'''
+        if not value_list:
+            return
+        self.c.executemany(self.get_insert_sql(
+            table_name, key, len(value_list[0]), insert_type), value_list)
+
+    def update(self, table_name: str, d: dict, query: 'Query' = None) -> None:
+        '''单表内行update单句sql语句'''
+        if not d:
+            return
+        sql, sql_list = self.get_update_sql(table_name, d, query)
+        self.c.execute(sql, sql_list)
+
+    def delete(self, table_name: str, query: 'Query' = None) -> None:
+        '''删除，query中只有query和fuzzy_query会被处理'''
+        sql, sql_list = self.get_delete_sql(table_name, query)
+        self.c.execute(sql, sql_list)
+
+    def get_table_info(self, table_name: str):
+        '''得到表结构，返回主键列表和字段名列表'''
+        pk = []
+        name = []
+
+        self.c.execute('''pragma table_info ("%s")''' % table_name)  # 这里无法参数化
+        x = self.c.fetchall()
+        if x:
+            for i in x:
+                name.append(i[1])
+                if i[5] != 0:
+                    pk.append(i[1])
+
+        return pk, name
+
+
+class DatabaseMigrator:
+
+    def __init__(self, c1_path: str, c2_path: str) -> None:
+        self.c1_path = c1_path
+        self.c2_path = c2_path
+
+    @staticmethod
+    def update_one_table(c1, c2, table_name: str) -> bool:
+        '''从c1向c2更新数据表，c1中存在的信息不变，即c2中的冲突信息会被覆盖'''
+        c1.execute(
+            '''select * from sqlite_master where type = 'table' and name = :a''', {'a': table_name})
+        c2.execute(
+            '''select * from sqlite_master where type = 'table' and name = :a''', {'a': table_name})
+        if not c1.fetchone() or not c2.fetchone():
+            return False
+
+        sql1 = Sql(c1)
+        sql2 = Sql(c2)
+        db1_pk, db1_name = sql1.get_table_info(table_name)
+        db2_pk, db2_name = sql2.get_table_info(table_name)
+        if db1_pk != db2_pk:
+            return False
+
+        sql2.insert_many(table_name, [], sql1.select(
+            table_name, list(filter(lambda x: x in db2_name, db1_name))), insert_type='replace')
+
+        return True
+
+    @staticmethod
+    def update_user_char_full(c) -> None:
+        '''用character表数据更新user_char_full'''
+        c.execute('''select character_id, max_level, is_uncapped from character''')
+        x = c.fetchall()
+        c.execute('''select user_id from user''')
+        y = c.fetchall()
+        c.execute('''delete from user_char_full''')
+        for i in x:
+            exp = 25000 if i[1] == 30 else 10000
+            c.executemany('''insert into user_char_full values(?,?,?,?,?,?)''', [
+                          (j[0], i[0], i[1], exp, i[2], 0) for j in y])
+
+    @staticmethod
+    def update_user_epilogue(c) -> None:
+        '''给用户添加epilogue包'''
+        c.execute('''select user_id from user''')
+        Sql(c).insert_many('user_item', [], [(i[0], 'epilogue', 'pack', 1)
+                                             for i in c.fetchall()], insert_type='ignore')
+
+    def update_database(self) -> None:
+        '''
+        将c1数据库不存在数据加入或覆盖到c2数据库上
+        对于c2，更新一些表，并用character数据更新user_char_full
+        '''
+        with Connect(self.c2_path) as c2:
+            with Connect(self.c1_path) as c1:
+                [self.update_one_table(c1, c2, i)
+                 for i in Constant.DATABASE_MIGRATE_TABLES]
+
+                if not Constant.UPDATE_WITH_NEW_CHARACTER_DATA:
+                    self.update_one_table(c1, c2, 'character')
+
+            self.update_user_char_full(c2)  # 更新user_char_full
+            self.update_user_epilogue(c2)  # 更新user的epilogue
+
+
+class MemoryDatabase:
+    conn = sqlite3.connect('file:arc_tmp?mode=memory&cache=shared', uri=True)
+
+    def __init__(self):
+        self.c = self.conn.cursor()
+        self.c.execute('''PRAGMA journal_mode = OFF''')
+        self.c.execute('''PRAGMA synchronous = 0''')
+        self.c.execute('''create table if not exists download_token(user_id int, 
+        song_id text,file_name text,token text,time int,primary key(user_id, song_id, file_name));''')
+        self.c.execute(
+            '''create index if not exists download_token_1 on download_token (song_id, file_name);''')
+        self.conn.commit()
+
+
+@register
+def atexit():
+    MemoryDatabase.conn.close()

latest version/core/user.py

@@ -3,15 +3,15 @@ import hashlib
 import time
 from os import urandom
 
-from setting import Config
-
 from .character import UserCharacter, UserCharacterList
+from .config_manager import Config
 from .constant import Constant
 from .error import (ArcError, DataExist, FriendError, InputError, NoAccess,
-                    NoData, UserBan)
+                    NoData, RateLimit, UserBan)
 from .item import UserItemList
+from .limiter import ArcLimiter
 from .score import Score
-from .sql import Connect
+from .sql import Query, Sql
 from .world import Map, UserMap, UserStamina
 
 
@@ -32,17 +32,17 @@ def code_get_id(c, user_code: str) -> int:
 
 class User:
     def __init__(self) -> None:
-        self.name = None
+        self.name: str = None
-        self.email = None
+        self.email: str = None
-        self.password = None
+        self.password: str = None
-        self.user_id = None
+        self.user_id: int = None
-        self.user_code = None
+        self.user_code: str = None
 
         self.join_date = None
         self.rating_ptt: int = None  # 100 times
 
-        self.ticket = None
+        self.ticket: int = None
-        self.world_rank_score = None
+        self.world_rank_score: int = None
         self.ban_flag = None
 
     @property
@@ -63,7 +63,7 @@ class UserRegister(User):
             if self.c.fetchone() == (0,):
                 self.name = name
             else:
-                raise DataExist('Username exists.', 101, -203)
+                raise DataExist('Username exists.', 101, -210)
 
         else:
             raise InputError('Username is invalid.')
@@ -76,16 +76,28 @@ class UserRegister(User):
 
     def set_email(self, email: str):
         # 邮箱格式懒得多判断
-        if 4 <= len(email) <= 32 and '@' in email and '.' in email:
+        if 4 <= len(email) <= 64 and '@' in email and '.' in email:
             self.c.execute(
                 '''select exists(select * from user where email = :email)''', {'email': email})
             if self.c.fetchone() == (0,):
                 self.email = email
             else:
-                raise DataExist('Email address exists.', 102, -204)
+                raise DataExist('Email address exists.', 102, -211)
         else:
             raise InputError('Email address is invalid.')
 
+    def set_user_code(self, user_code: str) -> None:
+        '''设置用户的user_code'''
+        if len(user_code) == 9 and user_code.isdigit():
+            self.c.execute(
+                '''select exists(select * from user where user_code = ?)''', (user_code, ))
+            if self.c.fetchone() == (0,):
+                self.user_code = user_code
+            else:
+                raise DataExist('User code exists.', 103, -212)
+        else:
+            raise InputError('User code is invalid.')
+
     def _build_user_code(self):
         # 生成9位的user_code，用的自然是随机
         from random import randint
@@ -130,8 +142,10 @@ class UserRegister(User):
 
     def register(self):
         now = int(time.time() * 1000)
-        self._build_user_code()
+        if self.user_code is None:
-        self._build_user_id()
+            self._build_user_code()
+        if self.user_id is None:
+            self._build_user_id()
         self._insert_user_char()
 
         self.c.execute('''insert into user(user_id, name, password, join_date, user_code, rating_ptt, 
@@ -144,6 +158,8 @@ class UserRegister(User):
 
 class UserLogin(User):
     # 密码和token的加密方式为 SHA-256
+    limiter = ArcLimiter(Config.GAME_LOGIN_RATE_LIMIT, 'game_login')
+
     def __init__(self, c) -> None:
         super().__init__()
         self.c = c
@@ -220,6 +236,9 @@ class UserLogin(User):
         if ip:
             self.set_ip(ip)
 
+        if not self.limiter.hit(name):
+            raise RateLimit('Too many login attempts.', 123, -203)
+
         self.c.execute('''select user_id, password, ban_flag from user where name = :name''', {
                        'name': self.name})
         x = self.c.fetchone()
@@ -276,7 +295,7 @@ class UserAuth(User):
 
 class UserInfo(User):
     def __init__(self, c, user_id=None) -> None:
-        super().__init__()
+        User.__init__(self)
         self.c = c
         self.user_id = user_id
         self.character = None
@@ -370,40 +389,42 @@ class UserInfo(User):
         self.favorite_character.select_character_uncap_condition(self)
         return self.favorite_character
 
+    @property
+    def friend_ids(self) -> list:
+        self.c.execute('''select user_id_other from friend where user_id_me = :user_id''', {
+            'user_id': self.user_id})
+        return self.c.fetchall()
+
     @property
     def friends(self) -> list:
         # 得到用户的朋友列表
         if self.__friends is None:
-            self.c.execute('''select user_id_other from friend where user_id_me = :user_id''', {
-                'user_id': self.user_id})
-            x = self.c.fetchall()
             s = []
-            if x != [] and x[0][0] is not None:
+            for i in self.friend_ids:
-                for i in x:
+                self.c.execute('''select exists(select * from friend where user_id_me = :x and user_id_other = :y)''',
-                    self.c.execute('''select exists(select * from friend where user_id_me = :x and user_id_other = :y)''',
+                               {'x': i[0], 'y': self.user_id})
-                                   {'x': i[0], 'y': self.user_id})
 
-                    is_mutual = True if self.c.fetchone() == (1,) else False
+                is_mutual = True if self.c.fetchone() == (1,) else False
 
-                    you = UserOnline(self.c, i[0])
+                you = UserOnline(self.c, i[0])
-                    you.select_user()
+                you.select_user()
-                    character = you.character if you.favorite_character is None else you.favorite_character
+                character = you.character if you.favorite_character is None else you.favorite_character
-                    character.select_character_uncap_condition(you)
+                character.select_character_uncap_condition(you)
 
-                    rating = you.rating_ptt if not you.is_hide_rating else -1
+                rating = you.rating_ptt if not you.is_hide_rating else -1
 
-                    s.append({
+                s.append({
-                        "is_mutual": is_mutual,
+                    "is_mutual": is_mutual,
-                        "is_char_uncapped_override": character.is_uncapped_override,
+                    "is_char_uncapped_override": character.is_uncapped_override,
-                        "is_char_uncapped": character.is_uncapped,
+                    "is_char_uncapped": character.is_uncapped,
-                        "is_skill_sealed": you.is_skill_sealed,
+                    "is_skill_sealed": you.is_skill_sealed,
-                        "rating": rating,
+                    "rating": rating,
-                        "join_date": you.join_date,
+                    "join_date": you.join_date,
-                        "character": character.character_id,
+                    "character": character.character_id,
-                        "recent_score": you.recent_score_list,
+                    "recent_score": you.recent_score_list,
-                        "name": you.name,
+                    "name": you.name,
-                        "user_id": you.user_id
+                    "user_id": you.user_id
-                    })
+                })
             s.sort(key=lambda item: item["recent_score"][0]["time_played"] if len(
                 item["recent_score"]) > 0 else 0, reverse=True)
             self.__friends = s
@@ -561,16 +582,8 @@ class UserInfo(User):
         self.stamina = UserStamina(self.c, self)
         self.stamina.set_value(x[0], x[1])
 
-    def select_user_about_character(self) -> None:
+    def from_list_about_character(self, x: list) -> None:
-        '''
+        '''从数据库user表获取搭档信息'''
-            查询user表有关角色的信息
-        '''
-        self.c.execute('''select name, character_id, is_skill_sealed, is_char_uncapped, is_char_uncapped_override, favorite_character from user where user_id = :a''', {
-            'a': self.user_id})
-        x = self.c.fetchone()
-        if not x:
-            raise NoData('No user.', 108, -3)
-
         self.name = x[0]
         self.character = UserCharacter(self.c, x[1], self)
         self.is_skill_sealed = x[2] == 1
@@ -579,6 +592,18 @@ class UserInfo(User):
         self.favorite_character = None if x[5] == - \
             1 else UserCharacter(self.c, x[5], self)
 
+    def select_user_about_character(self) -> None:
+        '''
+            查询user表有关搭档的信息
+        '''
+        self.c.execute('''select name, character_id, is_skill_sealed, is_char_uncapped, is_char_uncapped_override, favorite_character from user where user_id = :a''', {
+            'a': self.user_id})
+        x = self.c.fetchone()
+        if not x:
+            raise NoData('No user.', 108, -3)
+
+        self.from_list_about_character(x)
+
     def select_user_about_world_play(self) -> None:
         '''
             查询user表有关世界模式打歌的信息
@@ -616,9 +641,9 @@ class UserInfo(User):
 
     def update_global_rank(self) -> None:
         '''用户世界排名计算，有新增成绩则要更新'''
-        with Connect() as c2:
+
-            c2.execute('''select song_id, rating_ftr, rating_byn from chart''')
+        self.c.execute('''select song_id, rating_ftr, rating_byn from chart''')
-            x = c2.fetchall()
+        x = self.c.fetchall()
 
         song_list_ftr = [self.user_id]
         song_list_byn = [self.user_id]
@@ -720,3 +745,21 @@ class UserOnline(UserInfo):
         self.favorite_character = UserCharacter(self.c, character_id, self)
         self.c.execute('''update user set favorite_character = :a where user_id = :b''',
                        {'a': self.favorite_character.character_id, 'b': self.user_id})
+
+
+class UserChanger(UserInfo, UserRegister):
+
+    def __init__(self, c, user_id=None) -> None:
+        super().__init__(c, user_id)
+
+    def update_columns(self, columns: list = None, d: dict = None) -> None:
+        if columns is not None:
+            d = {}
+            for column in columns:
+                if column == 'password':
+                    d[column] = self.hash_pwd
+                else:
+                    d[column] = self.__dict__[column]
+
+        Sql(self.c).update('user', d, Query().from_args(
+            {'user_id': self.user_id}))

latest version/core/util.py

@@ -2,7 +2,7 @@ import hashlib
 import os
 
 
-def md5(code):
+def md5(code: str) -> str:
     # md5加密算法
     code = code.encode()
     md5s = hashlib.md5()
@@ -12,8 +12,8 @@ def md5(code):
     return codes
 
 
-def get_file_md5(file_path):
+def get_file_md5(file_path: str) -> str:
-    # 计算文件MD5
+    '''计算文件MD5'''
     if not os.path.isfile(file_path):
         return None
     myhash = hashlib.md5()
@@ -25,3 +25,17 @@ def get_file_md5(file_path):
             myhash.update(b)
 
     return myhash.hexdigest()
+
+
+def try_rename(path: str, new_path: str) -> str:
+    '''尝试重命名文件，并尝试避免命名冲突（在后面加自增数字），返回最终路径'''
+    final_path = new_path
+    if os.path.exists(new_path):
+        i = 1
+        while os.path.exists(new_path + str(i)):
+            i += 1
+
+        final_path = new_path + str(i)
+
+    os.rename(path, final_path)
+    return final_path

latest version/core/world.py

@@ -477,7 +477,7 @@ class WorldPlay:
 
         if self.character_bonus_progress is not None:
             # 猜的，为了让客户端正确显示，当然结果是没问题的
-            r['base_progress'] += self.character_bonus_progress
+            # r['base_progress'] += self.character_bonus_progress # 肯定不是这样的
             r['character_bonus_progress'] = self.character_bonus_progress
 
         if self.user_play.beyond_gauge == 0:
@@ -593,7 +593,7 @@ class WorldPlay:
 
     def after_climb(self) -> None:
         factory_dict = {'eto_uncap': self._eto_uncap, 'ayu_uncap': self._ayu_uncap,
-                        'luna_uncap': self._luna_uncap, 'skill_fatalis': self._skill_fatalis}
+                        'luna_uncap': self._luna_uncap, 'skill_fatalis': self._skill_fatalis, 'skill_amane': self._skill_amane}
         if self.character_used.skill_id_displayed in factory_dict:
             factory_dict[self.character_used.skill_id_displayed]()
 
@@ -666,3 +666,14 @@ class WorldPlay:
         self.user.world_mode_locked_end_ts = int(
             time()*1000) + Constant.SKILL_FATALIS_WORLD_LOCKED_TIME
         self.user.update_user_one_column('world_mode_locked_end_ts')
+
+    def _skill_amane(self) -> None:
+        '''
+        amane技能，起始格为限速或随机，成绩小于EX时，世界模式进度减半
+        偷懒在after_climb里面，需要重爬一次
+        '''
+        x: 'Step' = self.user.current_map.steps_for_climbing[0]
+        if ('randomsong' in x.step_type or 'speedlimit' in x.step_type) and self.user_play.song_grade < 5:
+            self.character_bonus_progress = -self.step_value / 2 / self.step_times
+            self.step_value = self.step_value / 2
+            self.user.current_map.reclimb(self.step_value)

latest version/database/database_initialize.py

@@ -1,605 +1,15 @@
-import sqlite3
+import os
-import time
+import sys
-import json
+from importlib import import_module
 
 # 数据库初始化文件，删掉arcaea_database.db文件后运行即可，谨慎使用
 
-ARCAEA_SERVER_VERSION = 'v2.10.0'
 
-
+def main(core_path: str = '../', db_path: str = './arcaea_database.db', init_folder_path: str = './init'):
-def main(path='./'):
+    sys.path.append(os.path.join(sys.path[0], core_path))
-    conn = sqlite3.connect(path+'arcaea_database.db')
+    sys.path.append(os.path.join(sys.path[0], core_path, './core/'))
-    c = conn.cursor()
+    init = import_module('init').DatabaseInit(db_path, init_folder_path)
-    c.execute('''create table if not exists config(id text primary key,
+    init.init()
-    value text
-    );''')
-    c.execute('''insert into config values("version", :a);''',
-              {'a': ARCAEA_SERVER_VERSION})
-    c.execute('''create table if not exists user(user_id int primary key,
-    name text unique,
-    password text not null,
-    join_date char(20),
-    user_code char(10),
-    rating_ptt int,
-    character_id int,
-    is_skill_sealed int,
-    is_char_uncapped int,
-    is_char_uncapped_override int,
-    is_hide_rating int,
-    song_id text,
-    difficulty int,
-    score int,
-    shiny_perfect_count int,
-    perfect_count int,
-    near_count int,
-    miss_count int,
-    health int,
-    modifier int,
-    time_played int,
-    clear_type int,
-    rating real,
-    favorite_character int,
-    max_stamina_notification_enabled int,
-    current_map text,
-    ticket int,
-    prog_boost int,
-    email text,
-    world_rank_score int,
-    ban_flag text,
-    next_fragstam_ts int,
-    max_stamina_ts int,
-    stamina int,
-    world_mode_locked_end_ts int
-    );''')
-    c.execute('''create table if not exists login(access_token text,
-    user_id int,
-    login_time int,
-    login_ip text,
-    login_device text,
-    primary key(access_token, user_id)
-    );''')
-    c.execute('''create table if not exists friend(user_id_me int,
-    user_id_other int,
-    primary key (user_id_me, user_id_other)
-    );''')
-    c.execute('''create table if not exists best_score(user_id int,
-    song_id text,
-    difficulty int,
-    score int,
-    shiny_perfect_count int,
-    perfect_count int,
-    near_count int,
-    miss_count int,
-    health int,
-    modifier int,
-    time_played int,
-    best_clear_type int,
-    clear_type int,
-    rating real,
-    primary key(user_id, song_id, difficulty)
-    );''')
-    c.execute('''create table if not exists user_char(user_id int,
-    character_id int,
-    level int,
-    exp real,
-    is_uncapped int,
-    is_uncapped_override int,
-    primary key(user_id, character_id)
-    );''')
-    c.execute('''create table if not exists user_char_full(user_id int,
-    character_id int,
-    level int,
-    exp real,
-    is_uncapped int,
-    is_uncapped_override int,
-    primary key(user_id, character_id)
-    );''')
-    c.execute('''create table if not exists character(character_id int primary key,
-    name text,
-    max_level int,
-    frag1 real,
-    prog1 real,
-    overdrive1 real,
-    frag20 real,
-    prog20 real,
-    overdrive20 real,
-    frag30 real,
-    prog30 real,
-    overdrive30 real,
-    skill_id text,
-    skill_unlock_level int,
-    skill_requires_uncap int,
-    skill_id_uncap text,
-    char_type int,
-    is_uncapped int
-    );''')
-    c.execute('''create table if not exists char_item(character_id int,
-    item_id text,
-    type text,
-    amount int,
-    primary key(character_id, item_id, type)
-    );''')
-    c.execute('''create table if not exists recent30(user_id int primary key,
-    r0 real,
-    song_id0 text,
-    r1 real,
-    song_id1 text,
-    r2 real,
-    song_id2 text,
-    r3 real,
-    song_id3 text,
-    r4 real,
-    song_id4 text,
-    r5 real,
-    song_id5 text,
-    r6 real,
-    song_id6 text,
-    r7 real,
-    song_id7 text,
-    r8 real,
-    song_id8 text,
-    r9 real,
-    song_id9 text,
-    r10 real,
-    song_id10 text,
-    r11 real,
-    song_id11 text,
-    r12 real,
-    song_id12 text,
-    r13 real,
-    song_id13 text,
-    r14 real,
-    song_id14 text,
-    r15 real,
-    song_id15 text,
-    r16 real,
-    song_id16 text,
-    r17 real,
-    song_id17 text,
-    r18 real,
-    song_id18 text,
-    r19 real,
-    song_id19 text,
-    r20 real,
-    song_id20 text,
-    r21 real,
-    song_id21 text,
-    r22 real,
-    song_id22 text,
-    r23 real,
-    song_id23 text,
-    r24 real,
-    song_id24 text,
-    r25 real,
-    song_id25 text,
-    r26 real,
-    song_id26 text,
-    r27 real,
-    song_id27 text,
-    r28 real,
-    song_id28 text,
-    r29 real,
-    song_id29 text
-    );''')
-    c.execute('''create table if not exists user_world(user_id int,
-    map_id text,
-    curr_position int,
-    curr_capture real,
-    is_locked int,
-    primary key(user_id, map_id)
-    );''')
-    c.execute('''create table if not exists songplay_token(token text primary key,
-    user_id int,
-    song_id text,
-    difficulty int,
-    course_id text,
-    course_state int,
-    course_score int,
-    course_clear_type int,
-    stamina_multiply int,
-    fragment_multiply int,
-    prog_boost_multiply int
-    );''')
-    c.execute('''create table if not exists download_token(user_id int,
-    song_id text,
-    file_name text,
-    token text,
-    time int,
-    primary key(user_id, song_id, file_name)
-    );''')
-    c.execute('''create table if not exists user_download(user_id int,
-    time int,
-    token text,
-    primary key(user_id, time, token)
-    );''')
-    c.execute('''create table if not exists item(item_id text,
-    type text,
-    is_available int,
-    _id text,
-    primary key(item_id, type)
-    );''')
-    c.execute('''create table if not exists user_item(user_id int,
-    item_id text,
-    type text,
-    amount int,
-    primary key(user_id, item_id, type)
-    );''')
-    c.execute('''create table if not exists purchase(purchase_name text primary key,
-    price int,
-    orig_price int,
-    discount_from int,
-    discount_to int,
-    discount_reason text
-    );''')
-    c.execute('''create table if not exists purchase_item(purchase_name text,
-    item_id text,
-    type text,
-    amount int,
-    primary key(purchase_name, item_id, type)
-    );''')
-    c.execute('''create table if not exists user_save(user_id int primary key,
-    scores_data text,
-    clearlamps_data text,
-    clearedsongs_data text,
-    unlocklist_data text,
-    installid_data text,
-    devicemodelname_data text,
-    story_data text,
-    createdAt int,
-    finalestate_data text
-    );''')
-    c.execute('''create table if not exists present(present_id text primary key,
-    expire_ts int,
-    description text
-    );''')
-    c.execute('''create table if not exists user_present(user_id int,
-    present_id text,
-    primary key(user_id, present_id)
-    );''')
-    c.execute('''create table if not exists present_item(present_id text,
-    item_id text,
-    type text,
-    amount int,
-    primary key(present_id, item_id, type)
-    );''')
-    c.execute('''create table if not exists chart(song_id text primary key,
-    name text,
-    rating_pst int,
-    rating_prs int,
-    rating_ftr int,
-    rating_byn int
-    );''')
-    c.execute('''create table if not exists redeem(code text primary key,
-    type int
-    );''')
-    c.execute('''create table if not exists user_redeem(user_id int,
-    code text,
-    primary key(user_id, code)
-    );''')
-    c.execute('''create table if not exists redeem_item(code text,
-    item_id text,
-    type text,
-    amount int,
-    primary key(code, item_id, type)
-    );''')
-
-    c.execute('''create table if not exists role(role_id int primary key,
-    role_name text,
-    caption text
-    );''')
-    c.execute('''create table if not exists user_role(user_id int,
-    role_id int,
-    primary key(user_id, role_id)
-    );''')
-    c.execute('''create table if not exists power(power_id int primary key,
-    power_name text,
-    caption text
-    );''')
-    c.execute('''create table if not exists role_power(role_id int,
-    power_id int,
-    primary key(role_id, power_id)
-    );''')
-    c.execute('''create table if not exists api_login(user_id int,
-    token text,
-    login_time int,
-    login_ip text,
-    primary key(user_id, token)
-    );''')
-    c.execute('''create table if not exists course(course_id text primary key,
-    course_name text,
-    dan_name text,
-    style int,
-    gauge_requirement text,
-    flag_as_hidden_when_requirements_not_met int,
-    can_start int
-    );''')
-    c.execute('''create table if not exists user_course(user_id int,
-    course_id text,
-    high_score int,
-    best_clear_type int,
-    primary key(user_id, course_id)
-    );''')
-    c.execute('''create table if not exists course_chart(course_id text,
-    song_id text,
-    difficulty int,
-    flag_as_hidden int,
-    song_index int,
-    primary key(course_id, song_index)
-    );''')
-    c.execute('''create table if not exists course_requirement(course_id text,
-    required_id text,
-    primary key(course_id, required_id)
-    );''')
-    c.execute('''create table if not exists course_item(course_id text,
-    item_id text,
-    type text,
-    amount int,
-    primary key(course_id, item_id, type)
-    );''')
-
-    c.execute(
-        '''create index best_score_1 on best_score (song_id, difficulty);''')  # 排名查询优化
-    c.execute(
-        '''create index download_token_1 on download_token (song_id, file_name);''')  # 下载token判断优化
-
-    # 搭档初始化
-    char = ['hikari', 'tairitsu', 'kou', 'sapphire', 'lethe', 'hikari&tairitsu(reunion)', 'Tairitsu(Axium)', 'Tairitsu(Grievous Lady)', 'stella', 'Hikari & Fisica', 'ilith', 'eto', 'luna', 'shirabe', 'Hikari(Zero)', 'Hikari(Fracture)', 'Hikari(Summer)', 'Tairitsu(Summer)', 'Tairitsu & Trin',
-            'ayu', 'Eto & Luna', 'yume', 'Seine & Hikari', 'saya', 'Tairitsu & Chuni Penguin', 'Chuni Penguin', 'haruna', 'nono', 'MTA-XXX', 'MDA-21', 'kanae', 'Hikari(Fantasia)', 'Tairitsu(Sonata)', 'sia', 'DORO*C', 'Tairitsu(Tempest)', 'brillante', 'Ilith(Summer)', 'etude', 'Alice & Tenniel', 'Luna & Mia', 'areus', 'seele', 'isabelle', 'mir', 'lagrange', 'linka', 'nami', 'Saya & Elizabeth', 'lily', 'kanae(midsummer)', 'alice&tenniel(minuet)', 'tairitsu(elegy)', 'marija', 'vita', 'hikari(fatalis)', 'saki', 'setsuna']
-
-    skill_id = ['gauge_easy', '', '', '', 'note_mirror', 'skill_reunion', '', 'gauge_hard', 'frag_plus_10_pack_stellights', 'gauge_easy|frag_plus_15_pst&prs', 'gauge_hard|fail_frag_minus_100', 'frag_plus_5_side_light', 'visual_hide_hp', 'frag_plus_5_side_conflict', 'challenge_fullcombo_0gauge', 'gauge_overflow', 'gauge_easy|note_mirror', 'note_mirror', 'visual_tomato_pack_tonesphere',
-                'frag_rng_ayu', 'gaugestart_30|gaugegain_70', 'combo_100-frag_1', 'audio_gcemptyhit_pack_groovecoaster', 'gauge_saya', 'gauge_chuni', 'kantandeshou', 'gauge_haruna', 'frags_nono', 'gauge_pandora', 'gauge_regulus', 'omatsuri_daynight', '', '', 'sometimes(note_mirror|frag_plus_5)', 'scoreclear_aa|visual_scoregauge', 'gauge_tempest', 'gauge_hard', 'gauge_ilith_summer', '', 'note_mirror|visual_hide_far', 'frags_ongeki', 'gauge_areus', 'gauge_seele', 'gauge_isabelle', 'gauge_exhaustion', 'skill_lagrange', 'gauge_safe_10', 'frags_nami', 'skill_elizabeth', 'skill_lily', 'skill_kanae_midsummer', '', '', 'visual_ghost_skynotes', 'skill_vita', 'skill_fatalis', 'frags_ongeki_slash', 'frags_ongeki_hard']
-
-    skill_id_uncap = ['', '', 'frags_kou', '', 'visual_ink', '', '', '', '', '', '', 'eto_uncap', 'luna_uncap', 'shirabe_entry_fee',
-                      '', '', '', '', '', 'ayu_uncap', '', 'frags_yume', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
-
-    skill_unlock_level = [0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0,
-                          0, 0, 0, 8, 0, 14, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 8, 0]
-
-    frag1 = [55, 55, 60, 50, 47, 79, 47, 57, 41, 22, 50, 54, 60, 56, 78, 42, 41, 61, 52, 50, 52, 32,
-             42, 55, 45, 58, 43, 0.5, 68, 50, 62, 45, 45, 52, 44, 27, 59, 0, 45, 50, 50, 47, 47, 61, 43, 42, 38, 25, 58, 50, 61, 45, 45, 38, 34, 27, 18, 56]
-
-    prog1 = [35, 55, 47, 50, 60, 70, 60, 70, 58, 45, 70, 45, 42, 46, 61, 67, 49, 44, 28, 45, 24, 46, 52,
-             59, 62, 33, 58, 25, 63, 69, 50, 45, 45, 51, 34, 70, 62, 70, 45, 32, 32, 61, 47, 47, 37, 42, 50, 50, 45, 41, 61, 45, 45, 58, 50, 130, 18, 57]
-
-    overdrive1 = [35, 55, 25, 50, 47, 70, 72, 57, 41, 7, 10, 32, 65, 31, 61, 53, 31, 47, 38, 12, 39, 18,
-                  48, 65, 45, 55, 44, 25, 46, 44, 33, 45, 45, 37, 25, 27, 50, 20, 45, 63, 21, 47, 61, 47, 65, 80, 38, 30, 49, 15, 34, 45, 45, 38, 67, 120, 44, 33]
-
-    frag20 = [78, 80, 90, 75, 70, 79, 70, 79, 65, 40, 50, 80, 90, 82, 0, 61, 67, 92, 85, 50, 86, 52,
-              65, 85, 67, 88, 64, 0.5, 95, 70, 95, 50, 80, 87, 71, 50, 85, 0, 80, 75, 50, 70, 70, 90, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 34, 85]
-
-    prog20 = [61, 80, 70, 75, 90, 70, 90, 102, 84, 78, 105, 67, 63, 68, 0, 99, 80, 66, 46, 83, 40, 73,
-              80, 90, 93, 50, 86, 78, 89, 98, 75, 80, 50, 64, 55, 100, 90, 110, 80, 50, 74, 90, 70, 70, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 34, 86]
-
-    overdrive20 = [61, 80, 47, 75, 70, 70, 95, 79, 65, 31, 50, 59, 90, 58, 0, 78, 50, 70, 62, 49, 64,
-                   46, 73, 95, 67, 84, 70, 78, 69, 70, 50, 80, 80, 63, 25, 50, 72, 55, 50, 95, 55, 70, 90, 70, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 84, 50]
-
-    frag30 = [88, 90, 100, 75, 80, 89, 70, 79, 65, 40, 50, 90, 100, 92, 0, 61, 67, 92, 85, 50, 86, 62,
-              65, 85, 67, 88, 74, 0.5, 105, 80, 95, 50, 80, 87, 71, 50, 95, 0, 80, 75, 50, 70, 80, 100, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 34, 85]
-
-    prog30 = [71, 90, 80, 75, 100, 80, 90, 102, 84, 78, 105, 77, 73, 78, 0, 99, 80, 66, 46, 93, 40, 83,
-              80, 90, 93, 50, 96, 88, 99, 108, 75, 80, 50, 64, 55, 100, 100, 110, 80, 50, 74, 90, 80, 80, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 34, 86]
-
-    overdrive30 = [71, 90, 57, 75, 80, 80, 95, 79, 65, 31, 50, 69, 100, 68, 0, 78, 50, 70, 62, 59, 64,
-                   56, 73, 95, 67, 84, 80, 88, 79, 80, 50, 80, 80, 63, 25, 50, 82, 55, 50, 95, 55, 70, 100, 80, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 84, 50]
-
-    char_type = [1, 0, 0, 0, 0, 0, 0, 2, 0, 1, 2, 0, 0, 0, 2, 3, 1, 0, 0, 0, 1,
-                 0, 0, 0, 0, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 2, 2, 2, 0, 0, 0, 2, 2, 2, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 2, 3, 0, 2]
-
-    char_core = {
-        0: [{'core_id': 'core_hollow', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
-        1: [{'core_id': 'core_hollow', 'amount': 5}, {'core_id': 'core_desolate', 'amount': 25}],
-        2: [{'core_id': 'core_hollow', 'amount': 5}, {'core_id': 'core_crimson', 'amount': 25}],
-        4: [{'core_id': 'core_ambivalent', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
-        13: [{'core_id': 'core_scarlet', 'amount': 30}],
-        21: [{'core_id': 'core_scarlet', 'amount': 30}],
-        26: [{'core_id': 'core_chunithm', 'amount': 15}],
-        27: [{'core_id': 'core_chunithm', 'amount': 15}],
-        28: [{'core_id': 'core_chunithm', 'amount': 15}],
-        29: [{'core_id': 'core_chunithm', 'amount': 15}],
-        36: [{'core_id': 'core_chunithm', 'amount': 15}],
-        42: [{'core_id': 'core_chunithm', 'amount': 15}],
-        43: [{'core_id': 'core_chunithm', 'amount': 15}],
-        11: [{'core_id': 'core_binary', 'amount': 25}, {'core_id': 'core_hollow', 'amount': 5}],
-        12: [{'core_id': 'core_binary', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
-        19: [{'core_id': 'core_colorful', 'amount': 30}]
-    }
-
-    for i in range(0, 58):
-        skill_requires_uncap = 1 if i == 2 else 0
-
-        if i in [0, 1, 2, 4, 13, 26, 27, 28, 29, 36, 21, 42, 43, 11, 12, 19, 5]:
-            sql = '''insert into character values(?,?,30,?,?,?,?,?,?,?,?,?,?,?,?,?,?,1)'''
-            c.execute(sql, (i, char[i], frag1[i], prog1[i], overdrive1[i], frag20[i], prog20[i], overdrive20[i],
-                            frag30[i], prog30[i], overdrive30[i], skill_id[i], skill_unlock_level[i], skill_requires_uncap, skill_id_uncap[i], char_type[i]))
-        else:
-            sql = '''insert into character values(?,?,20,?,?,?,?,?,?,?,?,?,?,?,?,?,?,0)'''
-            c.execute(sql, (i, char[i], frag1[i], prog1[i], overdrive1[i], frag20[i], prog20[i], overdrive20[i],
-                            frag30[i], prog30[i], overdrive30[i], skill_id[i], skill_unlock_level[i], skill_requires_uncap, skill_id_uncap[i], char_type[i]))
-
-    c.execute('''insert into character values(?,?,20,?,?,?,?,?,?,?,?,?,?,?,?,?,?,0)''', (99,
-                                                                                         'shirahime', 38, 33, 28, 66, 58, 50, 66, 58, 50, 'frags_preferred_song', 0, 0, '', 0))
-
-    for i in char_core:
-        for j in char_core[i]:
-            c.execute('''insert into char_item values(?,?,'core',?)''',
-                      (i, j['core_id'], j['amount']))
-    cores = ['core_hollow', 'core_desolate', 'core_chunithm', 'core_crimson',
-             'core_ambivalent', 'core_scarlet', 'core_groove', 'core_generic', 'core_binary', 'core_colorful', 'core_course_skip_purchase']
-
-    for i in cores:
-        c.execute('''insert into item values(?,"core",1,'')''', (i,))
-
-    world_songs = ["babaroque", "shadesoflight", "kanagawa", "lucifer", "anokumene", "ignotus", "rabbitintheblackroom", "qualia", "redandblue", "bookmaker", "darakunosono", "espebranch", "blacklotus", "givemeanightmare", "vividtheory", "onefr", "gekka", "vexaria3", "infinityheaven3", "fairytale3", "goodtek3", "suomi", "rugie", "faintlight", "harutopia", "goodtek", "dreaminattraction", "syro", "diode", "freefall", "grimheart", "blaster",
-                   "cyberneciacatharsis", "monochromeprincess", "revixy", "vector", "supernova", "nhelv", "purgatorium3", "dement3", "crossover", "guardina", "axiumcrisis", "worldvanquisher", "sheriruth", "pragmatism", "gloryroad", "etherstrike", "corpssansorganes", "lostdesire", "blrink", "essenceoftwilight", "lapis", "solitarydream", "lumia3", "purpleverse", "moonheart3", "glow", "enchantedlove", "take", "lifeispiano", "vandalism", "nexttoyou3", "lostcivilization3", "turbocharger", "bookmaker3", "laqryma3", "kyogenkigo", "hivemind", "seclusion", "quonwacca3", "bluecomet", "energysynergymatrix", "gengaozo", "lastendconductor3", "antithese3", "qualia3", "kanagawa3", "heavensdoor3", "pragmatism3", "nulctrl", "avril", "ddd", "merlin3", "omakeno3", "nekonote", "sanskia", 'altair', 'mukishitsu', 'trapcrow', 'redandblue3', 'ignotus3', 'singularity3', 'dropdead3', 'arcahv']
-    for i in world_songs:
-        c.execute('''insert into item values(?,"world_song",1,'')''', (i,))
-
-    world_unlocks = ["scenery_chap1", "scenery_chap2",
-                     "scenery_chap3", "scenery_chap4", "scenery_chap5", "scenery_chap6", "scenery_chap7"]
-    for i in world_unlocks:
-        c.execute('''insert into item values(?,"world_unlock",1,'')''', (i,))
-
-    course_banners = ['course_banner_' + str(i) for i in range(1, 12)]
-    for i in course_banners:
-        c.execute('''insert into item values(?,"course_banner",1,'')''', (i,))
-
-    c.execute('''insert into item values(?,?,?,?)''',
-              ('fragment', 'fragment', 1, ''))
-    c.execute('''insert into item values(?,?,?,?)''',
-              ('memory', 'memory', 1, ''))
-    c.execute('''insert into item values(?,?,?,?)''',
-              ('anni5tix', 'anni5tix', 1, ''))
-
-    def insert_items(c, items):
-        # 物品数据导入
-        for i in items:
-            if 'discount_from' not in i:
-                discount_from = -1
-            else:
-                discount_from = i['discount_from']
-            if 'discount_to' not in i:
-                discount_to = -1
-            else:
-                discount_to = i['discount_to']
-            if 'discount_reason' not in i:
-                discount_reason = ''
-            else:
-                discount_reason = i['discount_reason']
-            c.execute('''insert into purchase values(?,?,?,?,?,?)''',
-                      (i['name'], i['price'], i['orig_price'], discount_from, discount_to, discount_reason))
-            for j in i['items']:
-                if "_id" not in j:
-                    _id = ''
-                else:
-                    _id = j['_id']
-                c.execute(
-                    '''select exists(select * from item where item_id=?)''', (j['id'],))
-                if c.fetchone() == (0,):
-                    c.execute('''insert into item values(?,?,?,?)''',
-                              (j['id'], j['type'], j['is_available'], _id))
-                if 'amount' in j:
-                    amount = j['amount']
-                else:
-                    amount = 1
-                c.execute('''insert into purchase_item values(?,?,?,?)''',
-                          (i['name'], j['id'], j['type'], amount))
-
-    # item初始化
-    try:
-        f = open(path+'singles.json', 'r')
-        singles = json.load(f)
-        f.close()
-        insert_items(c, singles)
-    except:
-        pass
-
-    try:
-        f = open(path+'packs.json', 'r')
-        packs = json.load(f)
-        f.close()
-        insert_items(c, packs)
-        c.execute('''select exists(select * from item where item_id='epilogue')''')
-        if c.fetchone() == (0,):
-            c.execute('''insert into item values('epilogue','pack',1,'')''')
-    except:
-        pass
-
-    # course初始化
-    try:
-        f = open(path+'courses.json', 'r', encoding="utf-8")
-        courses = json.load(f)
-        f.close()
-    except:
-        courses = []
-
-    if courses:
-        if path == './':
-            # 有什么好办法吗
-            import sys
-            sys.path.append('..')
-        from core.course import Course
-        for i in courses:
-            x = Course(c).from_dict(i)
-            x.insert_all()
-
-        # api权限与权限组初始化
-    role = ['admin', 'user', 'selecter']
-    role_caption = ['管理员', '用户', '查询接口']
-
-    power = ['select', 'select_me', 'change', 'change_me', 'grant',
-             'grant_inf', 'select_song_rank', 'select_song_info']
-    power_caption = ['总体查询权限', '自我查询权限', '总体修改权限',
-                     '自我修改权限', '授权权限', '下级授权权限', '歌曲排行榜查询权限', '歌曲信息查询权限']
-
-    role_power = {'0': [0, 1, 3, 5, 6, 7],
-                  '1': [1, 3, 6, 7],
-                  '2': [0]
-                  }
-
-    for i in range(0, len(role)):
-        c.execute('''insert into role values(:a,:b,:c)''', {
-            'a': i, 'b': role[i], 'c': role_caption[i]})
-    for i in range(0, len(power)):
-        c.execute('''insert into power values(:a,:b,:c)''', {
-            'a': i, 'b': power[i], 'c': power_caption[i]})
-    for i in role_power:
-        for j in role_power[i]:
-            c.execute('''insert into role_power values(:a,:b)''',
-                      {'a': int(i), 'b': j})
-
-    conn.commit()
-    conn.close()
-
-    def arc_register(name: str):
-        def build_user_code(c):
-            return '123456789'
-
-        def build_user_id(c):
-            return 2000000
-
-        def insert_user_char(c, user_id):
-            # 为用户添加可用角色
-            c.execute('''insert into user_char values(?,?,?,?,?,?)''',
-                      (user_id, 0, 30, 25000, 1, 0))
-            c.execute('''insert into user_char values(?,?,?,?,?,?)''',
-                      (user_id, 1, 30, 25000, 1, 0))
-
-            c.execute(
-                '''select character_id, max_level, is_uncapped from character''')
-            x = c.fetchall()
-            if x:
-                for i in x:
-                    exp = 25000 if i[1] == 30 else 10000
-                    c.execute('''insert into user_char_full values(?,?,?,?,?,?)''',
-                              (user_id, i[0], i[1], exp, i[2], 0))
-
-        conn = sqlite3.connect(path+'arcaea_database.db')
-        c = conn.cursor()
-        hash_pwd = '41e5653fc7aeb894026d6bb7b2db7f65902b454945fa8fd65a6327047b5277fb'
-        c.execute(
-            '''select exists(select * from user where name = :name)''', {'name': name})
-        if c.fetchone() == (0,):
-            user_code = build_user_code(c)
-            user_id = build_user_id(c)
-            now = int(time.time() * 1000)
-            c.execute('''insert into user(user_id, name, password, join_date, user_code, rating_ptt,
-            character_id, is_skill_sealed, is_char_uncapped, is_char_uncapped_override, is_hide_rating, favorite_character, max_stamina_notification_enabled, current_map, ticket)
-            values(:user_id, :name, :password, :join_date, :user_code, 1250, 1, 0, 1, 0, 0, -1, 0, '', 114514)
-            ''', {'user_code': user_code, 'user_id': user_id, 'join_date': now, 'name': name, 'password': hash_pwd})
-            c.execute('''insert into recent30(user_id) values(:user_id)''', {
-                'user_id': user_id})
-            c.execute(
-                '''insert into best_score values(2000000,'vexaria',3,10000000,100,0,0,0,100,0,1599667200,3,3,10.8)''')
-            insert_user_char(c, user_id)
-            conn.commit()
-            conn.close()
-            return None
-        else:
-            conn.commit()
-            conn.close()
-            return None
-
-    arc_register('admin')
 
 
 if __name__ == '__main__':

latest version/database/init/arc_data.py

@@ -0,0 +1,86 @@
+class InitData:
+    char = ['hikari', 'tairitsu', 'kou', 'sapphire', 'lethe', 'hikari&tairitsu(reunion)', 'Tairitsu(Axium)', 'Tairitsu(Grievous Lady)', 'stella', 'Hikari & Fisica', 'ilith', 'eto', 'luna', 'shirabe', 'Hikari(Zero)', 'Hikari(Fracture)', 'Hikari(Summer)', 'Tairitsu(Summer)', 'Tairitsu & Trin',
+            'ayu', 'Eto & Luna', 'yume', 'Seine & Hikari', 'saya', 'Tairitsu & Chuni Penguin', 'Chuni Penguin', 'haruna', 'nono', 'MTA-XXX', 'MDA-21', 'kanae', 'Hikari(Fantasia)', 'Tairitsu(Sonata)', 'sia', 'DORO*C', 'Tairitsu(Tempest)', 'brillante', 'Ilith(Summer)', 'etude', 'Alice & Tenniel', 'Luna & Mia', 'areus', 'seele', 'isabelle', 'mir', 'lagrange', 'linka', 'nami', 'Saya & Elizabeth', 'lily', 'kanae(midsummer)', 'alice&tenniel(minuet)', 'tairitsu(elegy)', 'marija', 'vita', 'hikari(fatalis)', 'saki', 'setsuna', 'amane']
+
+    skill_id = ['gauge_easy', '', '', '', 'note_mirror', 'skill_reunion', '', 'gauge_hard', 'frag_plus_10_pack_stellights', 'gauge_easy|frag_plus_15_pst&prs', 'gauge_hard|fail_frag_minus_100', 'frag_plus_5_side_light', 'visual_hide_hp', 'frag_plus_5_side_conflict', 'challenge_fullcombo_0gauge', 'gauge_overflow', 'gauge_easy|note_mirror', 'note_mirror', 'visual_tomato_pack_tonesphere',
+                'frag_rng_ayu', 'gaugestart_30|gaugegain_70', 'combo_100-frag_1', 'audio_gcemptyhit_pack_groovecoaster', 'gauge_saya', 'gauge_chuni', 'kantandeshou', 'gauge_haruna', 'frags_nono', 'gauge_pandora', 'gauge_regulus', 'omatsuri_daynight', '', '', 'sometimes(note_mirror|frag_plus_5)', 'scoreclear_aa|visual_scoregauge', 'gauge_tempest', 'gauge_hard', 'gauge_ilith_summer', '', 'note_mirror|visual_hide_far', 'frags_ongeki', 'gauge_areus', 'gauge_seele', 'gauge_isabelle', 'gauge_exhaustion', 'skill_lagrange', 'gauge_safe_10', 'frags_nami', 'skill_elizabeth', 'skill_lily', 'skill_kanae_midsummer', '', '', 'visual_ghost_skynotes', 'skill_vita', 'skill_fatalis', 'frags_ongeki_slash', 'frags_ongeki_hard', 'skill_amane']
+
+    skill_id_uncap = ['', '', 'frags_kou', '', 'visual_ink', '', '', '', '', '', '', 'eto_uncap', 'luna_uncap', 'shirabe_entry_fee',
+                      '', '', '', '', '', 'ayu_uncap', '', 'frags_yume', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
+
+    skill_unlock_level = [0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0,
+                          0, 0, 0, 8, 0, 14, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 8, 0, 0]
+
+    frag1 = [55, 55, 60, 50, 47, 79, 47, 57, 41, 22, 50, 54, 60, 56, 78, 42, 41, 61, 52, 50, 52, 32,
+             42, 55, 45, 58, 43, 0.5, 68, 50, 62, 45, 45, 52, 44, 27, 59, 0, 45, 50, 50, 47, 47, 61, 43, 42, 38, 25, 58, 50, 61, 45, 45, 38, 34, 27, 18, 56, 47]
+
+    prog1 = [35, 55, 47, 50, 60, 70, 60, 70, 58, 45, 70, 45, 42, 46, 61, 67, 49, 44, 28, 45, 24, 46, 52,
+             59, 62, 33, 58, 25, 63, 69, 50, 45, 45, 51, 34, 70, 62, 70, 45, 32, 32, 61, 47, 47, 37, 42, 50, 50, 45, 41, 61, 45, 45, 58, 50, 130, 18, 57, 55]
+
+    overdrive1 = [35, 55, 25, 50, 47, 70, 72, 57, 41, 7, 10, 32, 65, 31, 61, 53, 31, 47, 38, 12, 39, 18,
+                  48, 65, 45, 55, 44, 25, 46, 44, 33, 45, 45, 37, 25, 27, 50, 20, 45, 63, 21, 47, 61, 47, 65, 80, 38, 30, 49, 15, 34, 45, 45, 38, 67, 120, 44, 33, 55]
+
+    frag20 = [78, 80, 90, 75, 70, 79, 70, 79, 65, 40, 50, 80, 90, 82, 0, 61, 67, 92, 85, 50, 86, 52,
+              65, 85, 67, 88, 64, 0.5, 95, 70, 95, 50, 80, 87, 71, 50, 85, 0, 80, 75, 50, 70, 70, 90, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 35, 85, 47]
+
+    prog20 = [61, 80, 70, 75, 90, 70, 90, 102, 84, 78, 105, 67, 63, 68, 0, 99, 80, 66, 46, 83, 40, 73,
+              80, 90, 93, 50, 86, 78, 89, 98, 75, 80, 50, 64, 55, 100, 90, 110, 80, 50, 74, 90, 70, 70, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 35, 86, 92]
+
+    overdrive20 = [61, 80, 47, 75, 70, 70, 95, 79, 65, 31, 50, 59, 90, 58, 0, 78, 50, 70, 62, 49, 64,
+                   46, 73, 95, 67, 84, 70, 78, 69, 70, 50, 80, 80, 63, 25, 50, 72, 55, 50, 95, 55, 70, 90, 70, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 85, 50, 92]
+
+    frag30 = [88, 90, 100, 75, 80, 89, 70, 79, 65, 40, 50, 90, 100, 92, 0, 61, 67, 92, 85, 50, 86, 62,
+              65, 85, 67, 88, 74, 0.5, 105, 80, 95, 50, 80, 87, 71, 50, 95, 0, 80, 75, 50, 70, 80, 100, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 35, 85, 47]
+
+    prog30 = [71, 90, 80, 75, 100, 80, 90, 102, 84, 78, 105, 77, 73, 78, 0, 99, 80, 66, 46, 93, 40, 83,
+              80, 90, 93, 50, 96, 88, 99, 108, 75, 80, 50, 64, 55, 100, 100, 110, 80, 50, 74, 90, 80, 80, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 35, 86, 92]
+
+    overdrive30 = [71, 90, 57, 75, 80, 80, 95, 79, 65, 31, 50, 69, 100, 68, 0, 78, 50, 70, 62, 59, 64,
+                   56, 73, 95, 67, 84, 80, 88, 79, 80, 50, 80, 80, 63, 25, 50, 82, 55, 50, 95, 55, 70, 100, 80, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 85, 50, 92]
+
+    char_type = [1, 0, 0, 0, 0, 0, 0, 2, 0, 1, 2, 0, 0, 0, 2, 3, 1, 0, 0, 0, 1,
+                 0, 0, 0, 0, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 2, 2, 2, 0, 0, 0, 2, 2, 2, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 2, 3, 0, 2, 2]
+
+    char_core = {
+        0: [{'core_id': 'core_hollow', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
+        1: [{'core_id': 'core_hollow', 'amount': 5}, {'core_id': 'core_desolate', 'amount': 25}],
+        2: [{'core_id': 'core_hollow', 'amount': 5}, {'core_id': 'core_crimson', 'amount': 25}],
+        4: [{'core_id': 'core_ambivalent', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
+        13: [{'core_id': 'core_scarlet', 'amount': 30}],
+        21: [{'core_id': 'core_scarlet', 'amount': 30}],
+        26: [{'core_id': 'core_chunithm', 'amount': 15}],
+        27: [{'core_id': 'core_chunithm', 'amount': 15}],
+        28: [{'core_id': 'core_chunithm', 'amount': 15}],
+        29: [{'core_id': 'core_chunithm', 'amount': 15}],
+        36: [{'core_id': 'core_chunithm', 'amount': 15}],
+        42: [{'core_id': 'core_chunithm', 'amount': 15}],
+        43: [{'core_id': 'core_chunithm', 'amount': 15}],
+        11: [{'core_id': 'core_binary', 'amount': 25}, {'core_id': 'core_hollow', 'amount': 5}],
+        12: [{'core_id': 'core_binary', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
+        19: [{'core_id': 'core_colorful', 'amount': 30}]
+    }
+
+    cores = ['core_hollow', 'core_desolate', 'core_chunithm', 'core_crimson',
+             'core_ambivalent', 'core_scarlet', 'core_groove', 'core_generic', 'core_binary', 'core_colorful', 'core_course_skip_purchase']
+
+    world_songs = ["babaroque", "shadesoflight", "kanagawa", "lucifer", "anokumene", "ignotus", "rabbitintheblackroom", "qualia", "redandblue", "bookmaker", "darakunosono", "espebranch", "blacklotus", "givemeanightmare", "vividtheory", "onefr", "gekka", "vexaria3", "infinityheaven3", "fairytale3", "goodtek3", "suomi", "rugie", "faintlight", "harutopia", "goodtek", "dreaminattraction", "syro", "diode", "freefall", "grimheart", "blaster",
+                   "cyberneciacatharsis", "monochromeprincess", "revixy", "vector", "supernova", "nhelv", "purgatorium3", "dement3", "crossover", "guardina", "axiumcrisis", "worldvanquisher", "sheriruth", "pragmatism", "gloryroad", "etherstrike", "corpssansorganes", "lostdesire", "blrink", "essenceoftwilight", "lapis", "solitarydream", "lumia3", "purpleverse", "moonheart3", "glow", "enchantedlove", "take", "lifeispiano", "vandalism", "nexttoyou3", "lostcivilization3", "turbocharger", "bookmaker3", "laqryma3", "kyogenkigo", "hivemind", "seclusion", "quonwacca3", "bluecomet", "energysynergymatrix", "gengaozo", "lastendconductor3", "antithese3", "qualia3", "kanagawa3", "heavensdoor3", "pragmatism3", "nulctrl", "avril", "ddd", "merlin3", "omakeno3", "nekonote", "sanskia", 'altair', 'mukishitsu', 'trapcrow', 'redandblue3', 'ignotus3', 'singularity3', 'dropdead3', 'arcahv', 'freefall3']
+
+    world_unlocks = ["scenery_chap1", "scenery_chap2",
+                     "scenery_chap3", "scenery_chap4", "scenery_chap5", "scenery_chap6", "scenery_chap7"]
+
+    course_banners = ['course_banner_' + str(i) for i in range(1, 12)]
+
+    role = ['system', 'admin', 'user', 'selecter']
+    role_caption = ['系统', '管理员', '用户', '查询接口']
+
+    power = ['system', 'select', 'select_me', 'change', 'change_me', 'grant',
+             'grant_inf', 'select_song_rank', 'select_song_info']
+    power_caption = ['系统权限', '总体查询权限', '自我查询权限', '总体修改权限',
+                     '自我修改权限', '授权权限', '下级授权权限', '歌曲排行榜查询权限', '歌曲信息查询权限']
+
+    role_power = {'system': power,
+                  'admin': ['select', 'select_me', 'change_me', 'grant_inf', 'select_song_rank', 'select_song_info'],
+                  'user': ['select_me', 'change_me', 'select_song_rank', 'select_song_info'],
+                  'selector': ['select']
+                  }

latest version/database/init/singles.json

@@ -1216,5 +1216,41 @@
 		],
 		"orig_price": 100,
 		"price": 100
+	},
+	{
+		"name": "freemyself",
+		"items": [
+			{
+				"type": "single",
+				"id": "freemyself",
+				"is_available": true
+			},
+			{
+				"type": "core",
+				"amount": 1,
+				"id": "core_generic",
+				"is_available": true
+			}
+		],
+		"orig_price": 100,
+		"price": 100
+	},
+	{
+		"name": "cocorocosmetic",
+		"items": [
+			{
+				"type": "single",
+				"id": "cocorocosmetic",
+				"is_available": true
+			},
+			{
+				"type": "core",
+				"amount": 1,
+				"id": "core_generic",
+				"is_available": true
+			}
+		],
+		"orig_price": 100,
+		"price": 100
 	}
 ]

latest version/database/init/tables.sql

@@ -0,0 +1,312 @@
+create table if not exists config(id text primary key, value text);
+create table if not exists user(user_id int primary key,
+name text unique,
+password text,
+join_date char(20),
+user_code char(10),
+rating_ptt int,
+character_id int,
+is_skill_sealed int,
+is_char_uncapped int,
+is_char_uncapped_override int,
+is_hide_rating int,
+song_id text,
+difficulty int,
+score int,
+shiny_perfect_count int,
+perfect_count int,
+near_count int,
+miss_count int,
+health int,
+modifier int,
+time_played int,
+clear_type int,
+rating real,
+favorite_character int,
+max_stamina_notification_enabled int,
+current_map text,
+ticket int,
+prog_boost int,
+email text,
+world_rank_score int,
+ban_flag text,
+next_fragstam_ts int,
+max_stamina_ts int,
+stamina int,
+world_mode_locked_end_ts int
+);
+create table if not exists login(access_token text,
+user_id int,
+login_time int,
+login_ip text,
+login_device text,
+primary key(access_token, user_id)
+);
+create table if not exists friend(user_id_me int,
+user_id_other int,
+primary key (user_id_me, user_id_other)
+);
+create table if not exists best_score(user_id int,
+song_id text,
+difficulty int,
+score int,
+shiny_perfect_count int,
+perfect_count int,
+near_count int,
+miss_count int,
+health int,
+modifier int,
+time_played int,
+best_clear_type int,
+clear_type int,
+rating real,
+primary key(user_id, song_id, difficulty)
+);
+create table if not exists user_char(user_id int,
+character_id int,
+level int,
+exp real,
+is_uncapped int,
+is_uncapped_override int,
+primary key(user_id, character_id)
+);
+create table if not exists user_char_full(user_id int,
+character_id int,
+level int,
+exp real,
+is_uncapped int,
+is_uncapped_override int,
+primary key(user_id, character_id)
+);
+create table if not exists character(character_id int primary key,
+name text,
+max_level int,
+frag1 real,
+prog1 real,
+overdrive1 real,
+frag20 real,
+prog20 real,
+overdrive20 real,
+frag30 real,
+prog30 real,
+overdrive30 real,
+skill_id text,
+skill_unlock_level int,
+skill_requires_uncap int,
+skill_id_uncap text,
+char_type int,
+is_uncapped int
+);
+create table if not exists char_item(character_id int,
+item_id text,
+type text,
+amount int,
+primary key(character_id, item_id, type)
+);
+create table if not exists recent30(user_id int primary key,
+r0 real,
+song_id0 text,
+r1 real,
+song_id1 text,
+r2 real,
+song_id2 text,
+r3 real,
+song_id3 text,
+r4 real,
+song_id4 text,
+r5 real,
+song_id5 text,
+r6 real,
+song_id6 text,
+r7 real,
+song_id7 text,
+r8 real,
+song_id8 text,
+r9 real,
+song_id9 text,
+r10 real,
+song_id10 text,
+r11 real,
+song_id11 text,
+r12 real,
+song_id12 text,
+r13 real,
+song_id13 text,
+r14 real,
+song_id14 text,
+r15 real,
+song_id15 text,
+r16 real,
+song_id16 text,
+r17 real,
+song_id17 text,
+r18 real,
+song_id18 text,
+r19 real,
+song_id19 text,
+r20 real,
+song_id20 text,
+r21 real,
+song_id21 text,
+r22 real,
+song_id22 text,
+r23 real,
+song_id23 text,
+r24 real,
+song_id24 text,
+r25 real,
+song_id25 text,
+r26 real,
+song_id26 text,
+r27 real,
+song_id27 text,
+r28 real,
+song_id28 text,
+r29 real,
+song_id29 text
+);
+create table if not exists user_world(user_id int,
+map_id text,
+curr_position int,
+curr_capture real,
+is_locked int,
+primary key(user_id, map_id)
+);
+create table if not exists songplay_token(token text primary key,
+user_id int,
+song_id text,
+difficulty int,
+course_id text,
+course_state int,
+course_score int,
+course_clear_type int,
+stamina_multiply int,
+fragment_multiply int,
+prog_boost_multiply int
+);
+create table if not exists item(item_id text,
+type text,
+is_available int,
+primary key(item_id, type)
+);
+create table if not exists user_item(user_id int,
+item_id text,
+type text,
+amount int,
+primary key(user_id, item_id, type)
+);
+create table if not exists purchase(purchase_name text primary key,
+price int,
+orig_price int,
+discount_from int,
+discount_to int,
+discount_reason text
+);
+create table if not exists purchase_item(purchase_name text,
+item_id text,
+type text,
+amount int,
+primary key(purchase_name, item_id, type)
+);
+create table if not exists user_save(user_id int primary key,
+scores_data text,
+clearlamps_data text,
+clearedsongs_data text,
+unlocklist_data text,
+installid_data text,
+devicemodelname_data text,
+story_data text,
+createdAt int,
+finalestate_data text
+);
+create table if not exists present(present_id text primary key,
+expire_ts int,
+description text
+);
+create table if not exists user_present(user_id int,
+present_id text,
+primary key(user_id, present_id)
+);
+create table if not exists present_item(present_id text,
+item_id text,
+type text,
+amount int,
+primary key(present_id, item_id, type)
+);
+create table if not exists chart(song_id text primary key,
+name text,
+rating_pst int,
+rating_prs int,
+rating_ftr int,
+rating_byn int
+);
+create table if not exists redeem(code text primary key,
+type int
+);
+create table if not exists user_redeem(user_id int,
+code text,
+primary key(user_id, code)
+);
+create table if not exists redeem_item(code text,
+item_id text,
+type text,
+amount int,
+primary key(code, item_id, type)
+);
+
+create table if not exists role(role_id text primary key,
+caption text
+);
+create table if not exists user_role(user_id int,
+role_id text,
+primary key(user_id, role_id)
+);
+create table if not exists power(power_id text primary key,
+caption text
+);
+create table if not exists role_power(role_id text,
+power_id text,
+primary key(role_id, power_id)
+);
+create table if not exists api_login(user_id int,
+token text,
+login_time int,
+login_ip text,
+primary key(user_id, token)
+);
+create table if not exists course(course_id text primary key,
+course_name text,
+dan_name text,
+style int,
+gauge_requirement text,
+flag_as_hidden_when_requirements_not_met int,
+can_start int
+);
+create table if not exists user_course(user_id int,
+course_id text,
+high_score int,
+best_clear_type int,
+primary key(user_id, course_id)
+);
+create table if not exists course_chart(course_id text,
+song_id text,
+difficulty int,
+flag_as_hidden int,
+song_index int,
+primary key(course_id, song_index)
+);
+create table if not exists course_requirement(course_id text,
+required_id text,
+primary key(course_id, required_id)
+);
+create table if not exists course_item(course_id text,
+item_id text,
+type text,
+amount int,
+primary key(course_id, item_id, type)
+);
+
+create index if not exists best_score_1 on best_score (song_id, difficulty);
+
+PRAGMA journal_mode = WAL;
+PRAGMA default_cache_size = 8000;

latest version/main.py

@@ -1,6 +1,20 @@
 # encoding: utf-8
 
 import os
+from importlib import import_module
+
+from core.config_manager import Config, ConfigManager
+
+if os.path.exists('config.py') or os.path.exists('config'):
+    # 导入用户自定义配置
+    ConfigManager.load(import_module('config').Config)
+
+if Config.DEPLOY_MODE == 'gevent':
+    # 异步
+    from gevent import monkey
+    monkey.patch_all()
+
+
 import sys
 from logging.config import dictConfig
 from multiprocessing import Process, set_start_method
@@ -10,27 +24,30 @@ from flask import Flask, make_response, request, send_from_directory
 
 import api
 import server
-import server.init
 import web.index
 import web.login
 from core.constant import Constant
-from core.download import (UserDownload, get_only_3_song_ids,
+from core.download import (UserDownload, initialize_songfile)
-                           initialize_songfile)
+from core.error import ArcError, NoAccess, RateLimit
-from core.error import ArcError
+from core.init import FileChecker
 from core.sql import Connect
 from server.func import error_return
-from setting import Config
 
 app = Flask(__name__)
 
-# from werkzeug.middleware.proxy_fix import ProxyFix
+if Config.USE_PROXY_FIX:
-# app.wsgi_app = ProxyFix(app.wsgi_app, x_proto=1, x_host=1)
+    # 代理修复
-# from flask_cors import CORS
+    from werkzeug.middleware.proxy_fix import ProxyFix
-# CORS(app, supports_credentials=True)
+    app.wsgi_app = ProxyFix(app.wsgi_app, x_proto=1, x_host=1)
+if Config.USE_CORS:
+    # 服务端跨域
+    from flask_cors import CORS
+    CORS(app, supports_credentials=True)
 
 
 os.chdir(sys.path[0])  # 更改工作路径，以便于愉快使用相对路径
 
+
 app.config.from_mapping(SECRET_KEY=Config.SECRET_KEY)
 app.config['SESSION_TYPE'] = 'filesystem'
 app.register_blueprint(web.login.bp)
@@ -55,22 +72,23 @@ def favicon():
 
 @app.route('/download/<path:file_path>', methods=['GET'])  # 下载
 def download(file_path):
-    with Connect() as c:
+    with Connect(in_memory=True) as c:
         try:
             x = UserDownload(c)
             x.token = request.args.get('t')
             x.song_id, x.file_name = file_path.split('/', 1)
             x.select_for_check()
             if x.is_limited:
-                raise ArcError(
+                raise RateLimit('You have reached the download limit.', 903)
-                    'You have reached the download limit.', 903, status=403)
             if not x.is_valid:
-                raise ArcError('Expired token.', status=403)
+                raise NoAccess('Expired token.')
-            x.insert_user_download()
+            x.download_hit()
-            # response = make_response()
+            if Config.DOWNLOAD_USE_NGINX_X_ACCEL_REDIRECT:
-            # response.headers['Content-Type'] = 'application/octet-stream'
+                # nginx X-Accel-Redirect
-            # response.headers['X-Accel-Redirect'] = '/nginx_download/' + file_path
+                response = make_response()
-            # return response
+                response.headers['Content-Type'] = 'application/octet-stream'
+                response.headers['X-Accel-Redirect'] = Config.NGINX_X_ACCEL_REDIRECT_PREFIX + file_path
+                return response
             return send_from_directory(Constant.SONG_FILE_FOLDER_PATH, file_path, as_attachment=True, conditional=True)
         except ArcError as e:
             if Config.ALLOW_WARNING_LOG:
@@ -79,10 +97,29 @@ def download(file_path):
     return error_return()
 
 
+if Config.DEPLOY_MODE == 'waitress':
+    # 给waitress加个日志
+    @app.after_request
+    def after_request(response):
+        app.logger.info(
+            f'{request.remote_addr} - - {request.method} {request.path} {response.status_code}')
+        return response
+
+
 def tcp_server_run():
-    if False:
+    if Config.DEPLOY_MODE == 'gevent':
+        # 异步 gevent WSGI server
+        host_port = (Config.HOST, Config.PORT)
+        app.logger.info('Running gevent WSGI server... (%s:%s)' % host_port)
         from gevent.pywsgi import WSGIServer
-        WSGIServer(("127.0.0.1", 5000), app).serve_forever()
+        WSGIServer(host_port, app, log=app.logger).serve_forever()
+    elif Config.DEPLOY_MODE == 'waitress':
+        # waitress WSGI server
+        import logging
+        from waitress import serve
+        logger = logging.getLogger('waitress')
+        logger.setLevel(logging.INFO)
+        serve(app, host=Config.HOST, port=Config.PORT)
     else:
         if Config.SSL_CERT and Config.SSL_KEY:
             app.run(Config.HOST, Config.PORT, ssl_context=(
@@ -145,7 +182,7 @@ def main():
 
     dictConfig(log_dict)
 
-    if not server.init.check_before_run(app):
+    if not FileChecker(app).check_before_run():
         app.logger.error('Something wrong. The server will not run.')
         input('Press ENTER key to exit.')
         sys.exit()
@@ -153,7 +190,6 @@ def main():
     app.logger.info("Start to initialize song data...")
     try:
         initialize_songfile()
-        get_only_3_song_ids()
         app.logger.info('Complete!')
     except:
         app.logger.warning('Initialization error!')

latest version/requirements.txt

@@ -1,2 +1,3 @@
 flask>=2.0.2
-cryptography>=35.0.0
+cryptography>=3.0.0
+limits>=2.7.0

latest version/server/__init__.py

@@ -1,5 +1,5 @@
+from core.config_manager import Config
 from flask import Blueprint
-from setting import Config
 
 from . import (auth, course, friend, multiplayer, others, present, purchase,
                score, user, world)

latest version/server/auth.py

@@ -1,11 +1,11 @@
 import base64
 from functools import wraps
 
+from core.config_manager import Config
 from core.error import ArcError, NoAccess
 from core.sql import Connect
 from core.user import UserAuth, UserLogin
 from flask import Blueprint, jsonify, request
-from setting import Config
 
 from .func import arc_try, error_return
 
@@ -54,11 +54,10 @@ def auth_required(request):
                 try:
                     user = UserAuth(c)
                     user.token = headers['Authorization'][7:]
-                    return view(user.token_get_id(), *args, **kwargs)
+                    user_id = user.token_get_id()
                 except ArcError as e:
                     return error_return(e)
-
+            return view(user_id, *args, **kwargs)
-            return error_return()
 
         return wrapped_view
     return decorator

latest version/server/course.py

@@ -1,6 +1,5 @@
 from core.constant import Constant
 from core.course import UserCourseList
-from core.error import ArcError
 from core.item import ItemCore
 from core.sql import Connect
 from core.user import UserOnline

latest version/server/func.py

@@ -1,9 +1,9 @@
 from functools import wraps
 from traceback import format_exc
 
+from core.config_manager import Config
 from core.error import ArcError
 from flask import current_app, jsonify
-from setting import Config
 
 default_error = ArcError('Unknown Error', status=500)
 

latest version/server/init.py

@@ -1,7 +1,8 @@
 import os
 from shutil import copy, copy2
+
 from core.sql import Connect
-from database.database_initialize import main, ARCAEA_SERVER_VERSION
+from database.database_initialize import ARCAEA_SERVER_VERSION, main
 from web.system import update_database
 
 
@@ -25,10 +26,6 @@ def check_before_run(app):
 
     f = True
 
-    if not os.path.exists('setting.py'):
-        app.logger.warning('File `setting.py` is missing.')
-        f = False
-
     if not os.path.exists('database'):
         app.logger.warning('Folder `database` is missing.')
         f = False

latest version/server/multiplayer.py

@@ -1,8 +1,8 @@
+from core.config_manager import Config
 from core.error import ArcError
 from core.linkplay import Player, RemoteMultiPlayer, Room
 from core.sql import Connect
 from flask import Blueprint, request
-from setting import Config
 
 from .auth import auth_required
 from .func import arc_try, success_return

latest version/server/others.py

@@ -2,7 +2,7 @@ import json
 from urllib.parse import parse_qs, urlparse
 
 from core.download import DownloadList
-from core.error import ArcError
+from core.error import RateLimit
 from core.sql import Connect
 from core.system import GameInfo
 from core.user import UserOnline
@@ -12,7 +12,7 @@ from werkzeug.datastructures import ImmutableMultiDict
 from .auth import auth_required
 from .func import arc_try, error_return, success_return
 from .present import present_info
-from .purchase import bundle_pack, bundle_bundle
+from .purchase import bundle_bundle, bundle_pack
 from .score import song_score_friend
 from .user import user_me
 from .world import world_all
@@ -29,13 +29,12 @@ def game_info():
 @auth_required(request)
 @arc_try
 def download_song(user_id):
-    with Connect() as c:
+    with Connect(in_memory=True) as c:
-        x = DownloadList(c, UserOnline(c, user_id))
+        x = DownloadList(c, UserOnline(None, user_id))
         x.song_ids = request.args.getlist('sid')
         x.url_flag = json.loads(request.args.get('url', 'true'))
-        x.clear_user_download()
+        if x.url_flag and x.is_limited:
-        if x.is_limited and x.url_flag:
+            raise RateLimit('You have reached the download limit.', 903)
-            raise ArcError('You have reached the download limit.', 903)
 
         x.add_songs()
         return success_return(x.urls)

latest version/server/user.py

@@ -1,11 +1,11 @@
 from core.character import UserCharacter
+from core.config_manager import Config
 from core.error import ArcError, NoAccess
 from core.item import ItemCore
 from core.save import SaveData
 from core.sql import Connect
 from core.user import User, UserLogin, UserOnline, UserRegister
 from flask import Blueprint, request
-from setting import Config
 
 from .auth import auth_required
 from .func import arc_try, success_return

latest version/web/index.py

@@ -2,7 +2,8 @@ import os
 import time
 
 import server.arcscore
-from core.download import initialize_songfile, get_only_3_song_ids
+from core.download import DownloadList, initialize_songfile
+from core.init import FileChecker
 from core.rank import RankList
 from core.sql import Connect
 from flask import Blueprint, flash, redirect, render_template, request, url_for
@@ -270,7 +271,8 @@ def update_database():
             file.save(os.path.join(UPLOAD_FOLDER, filename))
             flash('上传成功 Success upload.')
             try:
-                web.system.update_database()
+                FileChecker.update_database(
+                    os.path.join(UPLOAD_FOLDER, filename))
                 flash('数据更新成功 Success update data.')
             except:
                 flash('数据更新失败 Cannot update data.')
@@ -288,8 +290,7 @@ def update_database():
 def update_song_hash():
     # 更新数据库内谱面文件hash值
     try:
-        get_only_3_song_ids.cache_clear()
+        DownloadList.clear_all_cache()
-        get_only_3_song_ids()
         initialize_songfile()
         flash('数据刷新成功 Success refresh data.')
     except:
@@ -425,7 +426,7 @@ def all_character():
 def change_character():
     # 修改角色数据
     skill_ids = ['No_skill', 'gauge_easy', 'note_mirror', 'gauge_hard', 'frag_plus_10_pack_stellights', 'gauge_easy|frag_plus_15_pst&prs', 'gauge_hard|fail_frag_minus_100', 'frag_plus_5_side_light', 'visual_hide_hp', 'frag_plus_5_side_conflict', 'challenge_fullcombo_0gauge', 'gauge_overflow', 'gauge_easy|note_mirror', 'note_mirror', 'visual_tomato_pack_tonesphere',
-                 'frag_rng_ayu', 'gaugestart_30|gaugegain_70', 'combo_100-frag_1', 'audio_gcemptyhit_pack_groovecoaster', 'gauge_saya', 'gauge_chuni', 'kantandeshou', 'gauge_haruna', 'frags_nono', 'gauge_pandora', 'gauge_regulus', 'omatsuri_daynight', 'sometimes(note_mirror|frag_plus_5)', 'scoreclear_aa|visual_scoregauge', 'gauge_tempest', 'gauge_hard', 'gauge_ilith_summer', 'frags_kou', 'visual_ink', 'shirabe_entry_fee', 'frags_yume', 'note_mirror|visual_hide_far', 'frags_ongeki', 'gauge_areus', 'gauge_seele', 'gauge_isabelle', 'gauge_exhaustion', 'skill_lagrange', 'gauge_safe_10', 'frags_nami', 'skill_elizabeth', 'skill_lily', 'skill_kanae_midsummer', 'eto_uncap', 'luna_uncap', 'frags_preferred_song', 'visual_ghost_skynotes', 'ayu_uncap', 'skill_vita', 'skill_fatalis', 'skill_reunion', 'frags_ongeki_slash', 'frags_ongeki_hard']
+                 'frag_rng_ayu', 'gaugestart_30|gaugegain_70', 'combo_100-frag_1', 'audio_gcemptyhit_pack_groovecoaster', 'gauge_saya', 'gauge_chuni', 'kantandeshou', 'gauge_haruna', 'frags_nono', 'gauge_pandora', 'gauge_regulus', 'omatsuri_daynight', 'sometimes(note_mirror|frag_plus_5)', 'scoreclear_aa|visual_scoregauge', 'gauge_tempest', 'gauge_hard', 'gauge_ilith_summer', 'frags_kou', 'visual_ink', 'shirabe_entry_fee', 'frags_yume', 'note_mirror|visual_hide_far', 'frags_ongeki', 'gauge_areus', 'gauge_seele', 'gauge_isabelle', 'gauge_exhaustion', 'skill_lagrange', 'gauge_safe_10', 'frags_nami', 'skill_elizabeth', 'skill_lily', 'skill_kanae_midsummer', 'eto_uncap', 'luna_uncap', 'frags_preferred_song', 'visual_ghost_skynotes', 'ayu_uncap', 'skill_vita', 'skill_fatalis', 'skill_reunion', 'frags_ongeki_slash', 'frags_ongeki_hard', 'skill_amane']
     return render_template('web/changechar.html', skill_ids=skill_ids)
 
 
@@ -699,7 +700,7 @@ def change_item():
             c.execute(
                 '''select exists(select * from item where item_id=:a and type=:b)''', {'a': item_id, 'b': item_type})
             if c.fetchone() == (0,):
-                c.execute('''insert into item values(?,?,?,'')''',
+                c.execute('''insert into item values(?,?,?)''',
                           (item_id, item_type, is_available))
                 flash('物品添加成功 Successfully add the item.')
             else:

latest version/web/login.py

@@ -1,10 +1,10 @@
-#import sqlite3
-from flask import (Blueprint, flash, g, redirect,
-                   render_template, request, session, url_for)
 import functools
-from setting import Config
 import hashlib
 
+from core.config_manager import Config
+from flask import (Blueprint, flash, g, redirect, render_template, request,
+                   session, url_for)
+
 bp = Blueprint('login', __name__, url_prefix='/web')
 
 

latest version/web/system.py

@@ -1,9 +1,8 @@
-import os
-from core.sql import Connect
-import time
 import hashlib
+import time
 from random import Random
-from setting import Config
+
+from core.sql import Connect
 
 
 def int2b(x):
@@ -25,139 +24,6 @@ def random_str(randomlength=10):
     return s
 
 
-def get_table_info(c, table_name):
-    # 得到表结构，返回主键列表和字段名列表
-    pk = []
-    name = []
-
-    c.execute('''pragma table_info ("'''+table_name+'''")''')
-    x = c.fetchall()
-    if x:
-        for i in x:
-            name.append(i[1])
-            if i[5] != 0:
-                pk.append(i[1])
-
-    return pk, name
-
-
-def get_sql_select_table(table_name, get_field, where_field=[], where_value=[]):
-    # sql语句拼接，select ... from ... where ...
-    sql = 'select '
-    sql_dict = {}
-    if len(get_field) >= 2:
-        sql += get_field[0]
-        for i in range(1, len(get_field)):
-            sql += ',' + get_field[i]
-        sql += ' from ' + table_name
-    elif len(get_field) == 1:
-        sql += get_field[0] + ' from ' + table_name
-    else:
-        sql += '* from ' + table_name
-
-    if where_field and where_value:
-        sql += ' where '
-        sql += where_field[0] + '=:' + where_field[0]
-        sql_dict[where_field[0]] = where_value[0]
-        if len(where_field) >= 1:
-            for i in range(1, len(where_field)):
-                sql_dict[where_field[i]] = where_value[i]
-                sql += ' and ' + where_field[i] + '=:' + where_field[i]
-
-    sql += ' order by rowid'
-    return sql, sql_dict
-
-
-def get_sql_insert_table(table_name, field, value):
-    # sql语句拼接，insert into ...(...) values(...)
-    sql = 'insert into ' + table_name + '('
-    sql_dict = {}
-    sql2 = ''
-    if len(field) >= 2:
-        sql += field[0]
-        sql2 += ':' + field[0]
-        sql_dict[field[0]] = value[0]
-        for i in range(1, len(field)):
-            sql += ',' + field[i]
-            sql2 += ', :' + field[i]
-            sql_dict[field[i]] = value[i]
-
-        sql += ') values('
-
-    elif len(field) == 1:
-        sql += field[0] + ') values('
-        sql2 += ':' + field[0]
-        sql_dict[field[0]] = value[0]
-
-    else:
-        return 'error', {}
-
-    sql += sql2 + ')'
-
-    return sql, sql_dict
-
-
-def get_sql_delete_table(table_name, where_field=[], where_value=[]):
-    # sql语句拼接，delete from ... where ...
-    sql = 'delete from ' + table_name
-    sql_dict = {}
-
-    if where_field and where_value:
-        sql += ' where '
-        sql += where_field[0] + '=:' + where_field[0]
-        sql_dict[where_field[0]] = where_value[0]
-        if len(where_field) >= 1:
-            for i in range(1, len(where_field)):
-                sql_dict[where_field[i]] = where_value[i]
-                sql += ' and ' + where_field[i] + '=:' + where_field[i]
-
-    return sql, sql_dict
-
-
-def update_one_table(c1, c2, table_name):
-    # 从c1向c2更新数据表，c1中存在的信息不变
-    c1.execute(
-        '''select * from sqlite_master where type = 'table' and name = :a''', {'a': table_name})
-    c2.execute(
-        '''select * from sqlite_master where type = 'table' and name = :a''', {'a': table_name})
-    if not c1.fetchone() or not c2.fetchone():
-        return 'error'
-
-    db1_pk, db1_name = get_table_info(c1, table_name)
-    db2_pk, db2_name = get_table_info(c2, table_name)
-    if db1_pk != db2_pk:
-        return 'error'
-
-    field = []
-    for i in db1_name:
-        if i in db2_name:
-            field.append(i)
-
-    sql, sql_dict = get_sql_select_table(table_name, db1_pk)
-    c1.execute(sql)
-    x = c1.fetchall()
-    sql, sql_dict = get_sql_select_table(table_name, field)
-    c1.execute(sql)
-    y = c1.fetchall()
-    if x:
-        for i in range(0, len(x)):
-            sql, sql_dict = get_sql_select_table(
-                table_name, [], db1_pk, list(x[i]))
-            sql = 'select exists(' + sql + ')'
-            c2.execute(sql, sql_dict)
-
-            if c2.fetchone() == (1,):  # 如果c2里存在，先删除
-                sql, sql_dict = get_sql_delete_table(
-                    table_name, db1_pk, list(x[i]))
-                c2.execute(sql, sql_dict)
-
-            sql, sql_dict = get_sql_insert_table(
-                table_name, field, list(y[i]))
-            c2.execute(sql, sql_dict)
-
-    return None
-
-
 def update_user_char(c):
     # 用character数据更新user_char
     c.execute('''select character_id, max_level, is_uncapped from character''')
@@ -174,64 +40,6 @@ def update_user_char(c):
                           (j[0], i[0], i[1], exp, i[2], 0))
 
 
-def update_user_epilogue(c):
-    c.execute('''select user_id from user''')
-    x = c.fetchall()
-    for i in x:
-        c.execute(
-            '''select exists(select * from user_item where user_id=? and item_id=? and type='pack')''', (i[0], 'epilogue'))
-        if c.fetchone() == (0,):
-            c.execute('''insert into user_item values(?,?,'pack',1)''',
-                      (i[0], 'epilogue'))
-
-
-def update_database():
-    # 将old数据库不存在数据加入到新数据库上，并删除old数据库
-    # 对于arcaea_datebase.db，更新一些表，并用character数据更新user_char_full
-    if os.path.isfile("database/old_arcaea_database.db") and os.path.isfile("database/arcaea_database.db"):
-        with Connect('./database/old_arcaea_database.db') as c1:
-            with Connect() as c2:
-
-                update_one_table(c1, c2, 'user')
-                update_one_table(c1, c2, 'friend')
-                update_one_table(c1, c2, 'best_score')
-                update_one_table(c1, c2, 'recent30')
-                update_one_table(c1, c2, 'user_world')
-                update_one_table(c1, c2, 'item')
-                update_one_table(c1, c2, 'user_item')
-                update_one_table(c1, c2, 'purchase')
-                update_one_table(c1, c2, 'purchase_item')
-                update_one_table(c1, c2, 'user_save')
-                update_one_table(c1, c2, 'login')
-                update_one_table(c1, c2, 'present')
-                update_one_table(c1, c2, 'user_present')
-                update_one_table(c1, c2, 'present_item')
-                update_one_table(c1, c2, 'redeem')
-                update_one_table(c1, c2, 'user_redeem')
-                update_one_table(c1, c2, 'redeem_item')
-                update_one_table(c1, c2, 'role')
-                update_one_table(c1, c2, 'user_role')
-                update_one_table(c1, c2, 'power')
-                update_one_table(c1, c2, 'role_power')
-                update_one_table(c1, c2, 'api_login')
-                update_one_table(c1, c2, 'chart')
-                update_one_table(c1, c2, 'user_course')
-                update_one_table(c1, c2, 'course')
-                update_one_table(c1, c2, 'course_item')
-                update_one_table(c1, c2, 'course_chart')
-                update_one_table(c1, c2, 'course_requirement')
-
-                update_one_table(c1, c2, 'user_char')
-
-                if not Config.UPDATE_WITH_NEW_CHARACTER_DATA:
-                    update_one_table(c1, c2, 'character')
-
-                update_user_char(c2)  # 更新user_char_full
-                update_user_epilogue(c2)  # 更新user的epilogue
-
-        os.remove('database/old_arcaea_database.db')
-
-
 def unlock_all_user_item(c):
     # 解锁所有用户购买