Merge pull request #84 from Lost-MSth/dev

Update to v2.10.2

README.md

@@ -1,7 +1,9 @@
 # Arcaea-server
+
 一个微型的Arcaea本地服务器  A small local server for Arcaea
 
 ## 简介 Introduction
+
 这是基于Python以及Flask的微型本地Arcaea服务器，可以模拟游戏的主要功能。这可能是我第一次写这种大程序，若有不妥之处，敬请谅解。  
 
 本程序主要用于学习研究，不得用于任何商业行为，否则后果自负，这不是强制要求，只是一个提醒与警告。  
@@ -14,7 +16,9 @@ This procedure is mainly used for study and research, and shall not be used for
 > It looks stupid, but it works!
 
 ## 特性 Features
+
 有以下 We have：
+
 - 登录、注册 Login and registration
 - 多设备登录 Multi device login
 - 成绩上传 Score upload
@@ -24,7 +28,7 @@ This procedure is mainly used for study and research, and shall not be used for
 - 段位系统 Course system
 - Link Play
 - 好友系统 Friends
-- 数据同步 Data synchronization
+- 云端存档 Cloud save
 - 爬梯 Climbing steps
 - 自定义世界模式 Customizable World Mode
 - 自定义歌曲下载 Customizable songs download
@@ -40,60 +44,49 @@ This procedure is mainly used for study and research, and shall not be used for
 - 服务器日志 Server log
 
 没有以下 We don't have：
+
 - 服务器安全性保证 Server security assurance
 
 可能有问题 There may be problems：
+
 - Recent 30
 - 一些歌曲的解锁 Some songs' unlocking
 - 同设备多共存登录 Multiple app logins on the same device
 
 ## 说明 Statement
+
 只是很有趣，用处探索中。  
 It is just so interesting. What it can do is under exploration.
 
-
 ## 下载 Download
+
 [这里 Here](https://github.com/Lost-MSth/Arcaea-server/releases)
 
 [Arcaea-CN official](https://arcaea.lowiro.com/zh)  
-[Arcaea-lowi.ro](https://lowi.ro)  
-[Arcaea-RhyDown](https://rhydown.com)
+[Arcaea-Konmai Academy](https://616.sb)  
 
 ## 更新日志 Update log
+
 只保留最新版本 Only keep the latest version.
 
 > 提醒：更新时请注意保留原先的数据库，以防数据丢失。
 >
-> Tips: When updating, please keep the original database in case of data loss. 
+> Tips: When updating, please keep the original database in case of data loss.
 
+### Version 2.10.2
 
-### Version 2.10.1
-- 适用于Arcaea 4.1.4版本 For Arcaea 4.1.4
-- 新搭档 **天音** 已解锁 Unlock the character **Amane**.
-- 为**天音**技能提供支持 Add support for the skill of **Amane**.
-- 现在配置文件可以是含有部分选项的文件或模块 At present the setting file can be a module or a file with some of options.
-- 添加`waitress`和`gevent`的部署方案支持，并支持日志记录 Add deployment mode `waitress` and `gevent`, and add support for the info log recording of them.
-- 为`songlist`添加解析器以指定可下载的文件 Add a parser for `songlist` to specify downloadable files.
-- 重构数据库初始化和数据迁移部分 Code refactoring for database initialization and migration.
-- 限制用户下载频率将使用第三方限制器，替代数据库 Add a custom limiter and use it for limiting users' download rate instead of using database.
-  > 现在需要`limits`模块  
-  > Now `limits` module is required.
-- 为登录和API登录添加限制器 Add limiter for login and API login.
-- `sqlite3`数据库调整为WAL模式并增大缓存 Change journal mode to WAL and enlarge cache size for `sqlite3` database.
-- 将下载token放入内存中而不是文件数据库中 Put download token in memory database instead of filesystem database.
-- 加速`best_score`表多次查询，表现为歌曲排行榜查询性能提升 Accelerate multiple querying in `best_score` table, which results in performance improvement of song ranklist query.
-- 优化歌曲下载部分 Make some optimization for downloading songs.
-- **修复更新recent 10时可能出现的死循环问题 Fix a bug that there is a endless loop in calculating recent 10 updating.** (due to 6fcca179182775615115cdb255b3a8223831a8a0)
-- 修复课题模式成绩没有rating的问题 Fix a bug that scores in course mode cannot calculate rating.
-- 修正搭档数值 Fix a character's value.
-- 邮箱长度最大限制提升到64 Change the email max length to 64.
-- 新增API接口来获取用户身份与权限 Add a method of API for getting users' roles and powers.
-- 新增API接口来修改用户信息 Add a method of API to change the user's info.
-- 为API的`GET`请求添加`query`参数支持 Add support for the `query` param in API's `GET` requests.
-- 修复API的`best30`接口曲目无数据导致出错的问题 Fix a bug that `best30` of API cannot have scores whose songs are not in database.
-- 修复API的`recent30`接口用户成绩数量不足导致出错的问题 Fix a bug that users with no recent scores cannot get `recent30` via API.
+- 适用于Arcaea 4.1.7版本 For Arcaea 4.1.7
+- 新搭档 **红（冬日）** 已解锁 Unlock the character **Kou(Winter)**.
+- 新增记录数据库来记录全部的游玩历史分数 Add a log database to record all playing scores.
+- 新增设置选项，可选择阻止或接受unranked成绩 Add a config option that can be used to forbid unranked scores.
+- 为自定义异常添加简明的warning日志 Add brief warning logs for custom exceptions.
+- 修复flask应用启动前出现异常，日志无法正确地指出异常的问题 Fix a bug that if an exception is raised before flask app runs, logger will not work well.
+- 现在初始化文件中JSON文件可以是模块支持的其它编码格式 Now initial files can be other encoding types which are supported by JSON module.
+- `run.bat`在报错时会停下而不是一闪而过了 Make the `run.bat` script pause when meeting an error. #82
+- 新增API接口查询单谱排行 Add an API endpoint for getting the rank list of a song's chart. #81
 
 ## 运行环境与依赖 Running environment and requirements
+
 - Windows/Linux/Mac OS/Android...
 - Python >= 3.6
 - Flask module >= 2.0, Cryptography module >= 3.0.0, limits >= 2.7.0
@@ -106,34 +99,41 @@ It is just so interesting. What it can do is under exploration.
 -->
 
 ## 使用说明 Instruction for use
+
 [中文](https://github.com/Lost-MSth/Arcaea-server/wiki/%E4%BD%BF%E7%94%A8%E8%AF%B4%E6%98%8E)  
 [English](https://github.com/Lost-MSth/Arcaea-server/wiki/Instruction-for-use)
 
 ## 注意 Attentions
+
 [中文](https://github.com/Lost-MSth/Arcaea-server/wiki/%E6%B3%A8%E6%84%8F)  
 [English](https://github.com/Lost-MSth/Arcaea-server/wiki/Attentions)
 
-
 ## Q&A
+
 [中文/English](https://github.com/Lost-MSth/Arcaea-server/wiki/Q&A)
 
-
 ## 鸣谢 Thanks
-~~歌曲数据库来自 Using song database from
-[BotArcAPI releases](https://github.com/TheSnowfield/BotArcAPI/releases)~~
+
+歌曲数据库来自 Using song database from
+~~[BotArcAPI releases](https://github.com/TheSnowfield/BotArcAPI/releases)~~
+[ArcaeaSongDatabase](https://github.com/Arcaea-Infinity/ArcaeaSongDatabase)
+
 > 从v2.9开始不再提供歌曲数据  
 > Since v2.9, song data will not be provided.
 
 网站图标来自 Using favicon from [black fate - てんてん - pixiv](https://www.pixiv.net/artworks/82374369)
 
 ## 联系方式 Contact
+
 如有必要，可以联系本人 Contact me if necessary  
 邮箱 Email：th84292@foxmail.com
 
 ## 支持一下 Support me
+
 生活不易。 Life is not easy.  
 [支付宝 Alipay](https://github.com/Lost-MSth/Arcaea-server/blob/master/pic/Alipay.jpg)  
 [微信 WeChat](https://github.com/Lost-MSth/Arcaea-server/blob/master/pic/WeChat.png)
 
 ## 使用许可 Use license
+
 [MIT](LICENSE) © Lost

latest version/api/songs.py

@@ -1,4 +1,5 @@
-from core.error import NoData
+from core.error import NoData, InputError
+from core.rank import RankList
 from core.song import Song
 from core.sql import Connect, Query, Sql
 from flask import Blueprint, request
@@ -40,3 +41,27 @@ def songs_get(data, user):
             raise NoData(api_error_code=-2)
 
         return success_return([x.to_dict() for x in r])
+
+
+@bp.route('/<string:song_id>/<int:difficulty>/rank', methods=['GET'])
+@role_required(request, ['select', 'select_song_rank', 'select_song_rank_top'])
+@request_json_handle(request, optional_keys=['limit'])
+@api_try
+def songs_song_difficulty_rank_get(data, user, song_id, difficulty):
+    '''查询歌曲某个难度的成绩排行榜，和游戏内接口相似，只允许limit'''
+    if difficulty not in [0, 1, 2, 3]:
+        raise InputError('Difficulty must be 0, 1, 2 or 3')
+    limit = data.get('limit', 20)
+    if not isinstance(limit, int):
+        raise InputError('Limit must be int')
+    if user.role.only_has_powers(['select_song_rank_top'], ['select', 'select_song_rank']):
+        # 限制低权限只能查询前20名
+        if limit > 20 or limit < 0:
+            limit = 20
+    with Connect() as c:
+        rank_list = RankList(c)
+        rank_list.song.set_chart(song_id, difficulty)
+        rank_list.limit = limit
+        # 不检查歌曲是否存在，不存在返回的是空列表
+        rank_list.select_top()
+        return success_return(rank_list.to_dict_list())

latest version/core/api_user.py

@@ -34,6 +34,19 @@ class Role:
         '''判断role是否有power'''
         return any(power_id == i.power_id for i in self.powers)
 
+    def only_has_powers(self, power_ids: list, anti_power_ids: list = None) -> bool:
+        '''判断role是否全有power_ids里的power，且没有anti_power_ids里的任何一个power'''
+        flags = [False] * len(power_ids)
+        if anti_power_ids is None:
+            anti_power_ids = []
+        for i in self.powers:
+            if i.power_id in anti_power_ids:
+                return False
+            for j, k in enumerate(power_ids):
+                if i.power_id == k:
+                    flags[j] = True
+        return all(flags)
+
     def select_from_id(self, role_id: int = None) -> 'Role':
         '''用role_id查询role'''
         if role_id is not None:

latest version/core/config_manager.py

@@ -50,6 +50,8 @@ class Config:
     ALLOW_LOGIN_SAME_DEVICE = False
     ALLOW_BAN_MULTIDEVICE_USER_AUTO = True
 
+    ALLOW_SCORE_WITH_NO_SONG = True
+
     ALLOW_INFO_LOG = False
     ALLOW_WARNING_LOG = False
 
@@ -77,6 +79,7 @@ class Config:
     SQLITE_DATABASE_PATH = './database/arcaea_database.db'
     SQLITE_DATABASE_BACKUP_FOLDER_PATH = './database/backup/'
     DATABASE_INIT_PATH = './database/init/'
+    SQLITE_LOG_DATABASE_PATH = './database/arcaea_log.db'
 
     GAME_LOGIN_RATE_LIMIT = '30/5 minutes'
     API_LOGIN_RATE_LIMIT = '10/5 minutes'

latest version/core/constant.py

@@ -1,6 +1,6 @@
 from .config_manager import Config
 
-ARCAEA_SERVER_VERSION = 'v2.10.1'
+ARCAEA_SERVER_VERSION = 'v2.10.2'
 
 
 class Constant:
@@ -35,6 +35,7 @@ class Constant:
     SONG_FILE_FOLDER_PATH = Config.SONG_FILE_FOLDER_PATH
     SONGLIST_FILE_PATH = Config.SONGLIST_FILE_PATH
     SQLITE_DATABASE_PATH = Config.SQLITE_DATABASE_PATH
+    SQLITE_LOG_DATABASE_PATH = Config.SQLITE_LOG_DATABASE_PATH
 
     DOWNLOAD_TIMES_LIMIT = Config.DOWNLOAD_TIMES_LIMIT
     DOWNLOAD_TIME_GAP_LIMIT = Config.DOWNLOAD_TIME_GAP_LIMIT

latest version/core/download.py

@@ -20,15 +20,6 @@ def get_song_file_md5(song_id: str, file_name: str) -> str:
     return get_file_md5(path)
 
 
-def initialize_songfile():
-    '''初始化歌曲数据的md5信息'''
-    get_song_file_md5.cache_clear()
-    x = DownloadList()
-    x.url_flag = False
-    x.add_songs()
-    del x
-
-
 class SonglistParser:
     '''songlist文件解析器'''
 
@@ -51,7 +42,7 @@ class SonglistParser:
         self.parse()
 
     @staticmethod
-    def is_available_file(song_id: str, file_name: str) -> list:
+    def is_available_file(song_id: str, file_name: str) -> bool:
         '''判断文件是否允许被下载'''
         if song_id not in SonglistParser.songs:
             # songlist没有，则只限制文件名
@@ -189,13 +180,22 @@ class DownloadList(UserDownload):
         self.downloads: list = []
         self.urls: dict = {}
 
+    @classmethod
+    def initialize_cache(cls) -> None:
+        '''初始化歌曲数据缓存，包括md5、文件目录遍历、解析songlist'''
+        SonglistParser()
+        x = cls()
+        x.url_flag = False
+        x.add_songs()
+        del x
+
     @staticmethod
-    def clear_all_cache():
+    def clear_all_cache() -> None:
         '''清除所有歌曲文件有关缓存'''
         get_song_file_md5.cache_clear()
         DownloadList.get_one_song_file_names.cache_clear()
         DownloadList.get_all_song_ids.cache_clear()
-        SonglistParser()
+        SonglistParser.songs = {}
 
     def clear_download_token(self) -> None:
         '''清除过期下载链接'''

latest version/core/init.py

@@ -4,11 +4,12 @@ from importlib import import_module
 from json import load
 from shutil import copy, copy2
 from time import time
+from traceback import format_exc
 
 from core.config_manager import Config
 from core.constant import ARCAEA_SERVER_VERSION
 from core.course import Course
-from core.download import SonglistParser
+from core.download import DownloadList
 from core.purchase import Purchase
 from core.sql import Connect, DatabaseMigrator, MemoryDatabase
 from core.user import UserRegister
@@ -90,10 +91,10 @@ class DatabaseInit:
         self.c.execute('''insert into item values(?,?,?)''',
                        ('anni5tix', 'anni5tix', 1))
 
-        with open(self.pack_path, 'r') as f:
+        with open(self.pack_path, 'rb') as f:
             self.insert_purchase_item(load(f))
 
-        with open(self.single_path, 'r') as f:
+        with open(self.single_path, 'rb') as f:
             self.insert_purchase_item(load(f))
 
         self.c.execute(
@@ -105,7 +106,7 @@ class DatabaseInit:
     def course_init(self) -> None:
         '''初始化课题信息'''
         courses = []
-        with open(self.course_path, 'r', encoding='utf-8') as f:
+        with open(self.course_path, 'rb') as f:
             courses = load(f)
         for i in courses:
             x = Course(self.c).from_dict(i)
@@ -157,35 +158,73 @@ class DatabaseInit:
             self.admin_init()
 
 
-class FileChecker:
+class LogDatabaseInit:
+    def __init__(self, db_path: str = Config.SQLITE_LOG_DATABASE_PATH, init_folder_path: str = Config.DATABASE_INIT_PATH) -> None:
+        self.db_path = db_path
+        self.init_folder_path = init_folder_path
+        self.c = None
 
-    def __init__(self, app=None):
-        self.app = app
+    @property
+    def sql_path(self) -> str:
+        return os.path.join(self.init_folder_path, 'log_tables.sql')
+
+    def table_init(self) -> None:
+        '''初始化数据库结构'''
+        with open(self.sql_path, 'r') as f:
+            self.c.executescript(f.read())
+        self.c.execute(
+            '''insert into cache values("version", :a, -1);''', {'a': ARCAEA_SERVER_VERSION})
+
+    def init(self) -> None:
+        with Connect(self.db_path) as c:
+            self.c = c
+            self.table_init()
+
+
+class FileChecker:
+    '''文件检查及初始化类'''
+
+    def __init__(self, logger=None):
+        self.logger = logger
 
     def check_file(self, file_path: str) -> bool:
         f = os.path.isfile(file_path)
         if not f:
-            self.app.logger.warning('File `%s` is missing.' % file_path)
+            self.logger.warning('File `%s` is missing.' % file_path)
         return f
 
     def check_folder(self, folder_path: str) -> bool:
         f = os.path.isdir(folder_path)
         if not f:
-            self.app.logger.warning('Folder `%s` is missing.' % folder_path)
+            self.logger.warning('Folder `%s` is missing.' % folder_path)
         return f
 
     def check_update_database(self) -> bool:
+        if not self.check_file(Config.SQLITE_LOG_DATABASE_PATH):
+            # 新建日志数据库
+            try:
+                self.logger.info(
+                    f'Try to new the file {Config.SQLITE_LOG_DATABASE_PATH}')
+                LogDatabaseInit().init()
+                self.logger.info(
+                    f'Success to new the file {Config.SQLITE_LOG_DATABASE_PATH}')
+            except Exception as e:
+                self.logger.error(format_exc())
+                self.logger.error(
+                    f'Failed to new the file {Config.SQLITE_LOG_DATABASE_PATH}')
+                return False
         if not self.check_file(Config.SQLITE_DATABASE_PATH):
             # 新建数据库
             try:
-                self.app.logger.info(
+                self.logger.info(
                     'Try to new the file `%s`.' % Config.SQLITE_DATABASE_PATH)
                 DatabaseInit().init()
-                self.app.logger.info(
+                self.logger.info(
                     'Success to new the file `%s`.' % Config.SQLITE_DATABASE_PATH)
-            except:
-                self.app.logger.warning(
-                    'Fail to new the file `%s`.' % Config.SQLITE_DATABASE_PATH)
+            except Exception as e:
+                self.logger.error(format_exc())
+                self.logger.warning(
+                    'Failed to new the file `%s`.' % Config.SQLITE_DATABASE_PATH)
                 return False
         else:
             # 检查更新
@@ -197,10 +236,10 @@ class FileChecker:
                     x = None
             # 数据库自动更新，不强求
             if not x or x[0] != ARCAEA_SERVER_VERSION:
-                self.app.logger.warning(
+                self.logger.warning(
                     'Maybe the file `%s` is an old version.' % Config.SQLITE_DATABASE_PATH)
                 try:
-                    self.app.logger.info(
+                    self.logger.info(
                         'Try to update the file `%s`.' % Config.SQLITE_DATABASE_PATH)
 
                     if not os.path.isdir(Config.SQLITE_DATABASE_BACKUP_FOLDER_PATH):
@@ -224,11 +263,12 @@ class FileChecker:
                     DatabaseInit().init()
                     self.update_database(temp_path)
 
-                    self.app.logger.info(
+                    self.logger.info(
                         'Success to update the file `%s`.' % Config.SQLITE_DATABASE_PATH)
 
-                except ValueError:
-                    self.app.logger.warning(
+                except Exception as e:
+                    self.logger.error(format_exc())
+                    self.logger.warning(
                         'Fail to update the file `%s`.' % Config.SQLITE_DATABASE_PATH)
 
         return True
@@ -240,9 +280,20 @@ class FileChecker:
             DatabaseMigrator(old_path, new_path).update_database()
             os.remove(old_path)
 
+    def check_song_file(self) -> bool:
+        '''检查song有关文件并初始化缓存'''
+        f = self.check_folder(Config.SONG_FILE_FOLDER_PATH)
+        self.logger.info("Start to initialize song data...")
+        try:
+            DownloadList.initialize_cache()
+            self.logger.info('Complete!')
+        except Exception as e:
+            self.logger.error(format_exc())
+            self.logger.warning('Initialization error!')
+            f = False
+        return f
+
     def check_before_run(self) -> bool:
         '''运行前检查，返回布尔值'''
-        # TODO: try
         MemoryDatabase()  # 初始化内存数据库
-        SonglistParser()  # 解析songlist
-        return self.check_folder(Config.SONG_FILE_FOLDER_PATH) & self.check_update_database()
+        return self.check_song_file() & self.check_update_database()

latest version/core/operation.py

@@ -0,0 +1,67 @@
+from .sql import Connect, Sql
+from .score import Score
+from .download import DownloadList
+
+
+class BaseOperation:
+    name: str = None
+
+    def __init__(self):
+        pass
+
+    def __call__(self, *args, **kwargs):
+        return self.run(*args, **kwargs)
+
+    def run(self, *args, **kwargs):
+        raise NotImplementedError
+
+
+class RefreshAllScoreRating(BaseOperation):
+    '''
+        刷新所有成绩的评分
+    '''
+    name = 'refresh_all_score_rating'
+
+    def run(self):
+        # 追求效率，不用Song类，尽量不用对象
+        # 但其实还是很慢
+        with Connect() as c:
+            c.execute(
+                '''select song_id, rating_pst, rating_prs, rating_ftr, rating_byn from chart''')
+            x = c.fetchall()
+
+            songs = [i[0] for i in x]
+            c.execute(
+                f'''update best_score set rating=0 where song_id not in ({','.join(['?']*len(songs))})''', songs)
+
+            for i in x:
+                for j in range(0, 4):
+                    defnum = -10  # 没在库里的全部当做定数-10
+                    if i[j+1] is not None and i[j+1] > 0:
+                        defnum = float(i[j+1]) / 10
+
+                    c.execute('''select user_id, score from best_score where song_id=:a and difficulty=:b''', {
+                              'a': i[0], 'b': j})
+                    y = c.fetchall()
+                    values = []
+                    where_values = []
+                    for k in y:
+                        ptt = Score.calculate_rating(defnum, k[1])
+                        if ptt < 0:
+                            ptt = 0
+                        values.append((ptt,))
+                        where_values.append((k[0], i[0], j))
+                    if values:
+                        Sql(c).update_many('best_score', ['rating'], values, [
+                            'user_id', 'song_id', 'difficulty'], where_values)
+
+
+class RefreshSongFileCache(BaseOperation):
+    '''
+        刷新歌曲文件缓存，包括文件hash缓存重建、文件目录重遍历、songlist重解析
+    '''
+    name = 'refresh_song_file_cache'
+
+    def run(self):
+        DownloadList.clear_all_cache()
+        DownloadList.initialize_cache()

latest version/core/score.py

@@ -7,7 +7,7 @@ from .course import CoursePlay
 from .error import NoData, StaminaNotEnough
 from .item import ItemCore
 from .song import Chart
-from .sql import Query, Sql
+from .sql import Connect, Query, Sql
 from .util import md5
 from .world import WorldPlay
 
@@ -103,7 +103,7 @@ class Score:
 
     @staticmethod
     def calculate_rating(defnum: int, score: int) -> float:
-        '''计算rating，谱面定数小于等于0视为Unrank，这里的defnum = Chart const'''
+        '''计算rating，谱面定数小于等于0视为Unrank，返回值会为-1，这里的defnum = Chart const'''
         if not defnum or defnum <= 0:
             # 谱面没定数或者定数小于等于0被视作Unrank
             return -1
@@ -408,6 +408,12 @@ class UserPlay(UserScore):
 
         self.ptt.insert_recent_30()
 
+    def record_score(self) -> None:
+        '''向log数据库记录分数，请注意列名不同'''
+        with Connect(Constant.SQLITE_LOG_DATABASE_PATH) as c2:
+            c2.execute('''insert into user_score values(?,?,?,?,?,?,?,?,?,?,?,?,?)''', (self.user.user_id, self.song.song_id, self.song.difficulty, self.time_played,
+                                                                                        self.score, self.shiny_perfect_count, self.perfect_count, self.near_count, self.miss_count, self.health, self.modifier, self.clear_type, self.rating))
+
     def upload_score(self) -> None:
         '''上传分数，包括user的recent更新，best更新，recent30更新，世界模式计算'''
         self.get_play_state()
@@ -420,6 +426,9 @@ class UserPlay(UserScore):
 
         self.time_played = int(time())
 
+        # 记录分数
+        self.record_score()
+
         # recent更新
         self.c.execute('''update user set song_id = :b, difficulty = :c, score = :d, shiny_perfect_count = :e, perfect_count = :f, near_count = :g, miss_count = :h, health = :i, modifier = :j, clear_type = :k, rating = :l, time_played = :m  where user_id = :a''', {
             'a': self.user.user_id, 'b': self.song.song_id, 'c': self.song.difficulty, 'd': self.score, 'e': self.shiny_perfect_count, 'f': self.perfect_count, 'g': self.near_count, 'h': self.miss_count, 'i': self.health, 'j': self.modifier, 'k': self.clear_type, 'l': self.rating, 'm': self.time_played * 1000})

latest version/core/song.py

@@ -1,4 +1,5 @@
 from .error import NoData
+from .config_manager import Config
 
 
 class Chart:
@@ -33,8 +34,10 @@ class Chart:
             '''select rating_pst, rating_prs, rating_ftr, rating_byn from chart where song_id=:a''', {'a': self.song_id})
         x = self.c.fetchone()
         if x is None:
-            self.defnum = -10
-            # raise NoData('The song `%s` does not exist.' % self.song_id)
+            if Config.ALLOW_SCORE_WITH_NO_SONG:
+                self.defnum = -10
+            else:
+                raise NoData(f'The song `{self.song_id}` does not exist.', 120)
         else:
             self.defnum = x[self.difficulty]
 

latest version/core/sql.py

@@ -2,16 +2,15 @@ import sqlite3
 import traceback
 from atexit import register
 
-from flask import current_app
-
 from .constant import Constant
 from .error import ArcError, InputError
 
 
 class Connect:
     # 数据库连接类，上下文管理
+    logger = None
 
-    def __init__(self, file_path: str = Constant.SQLITE_DATABASE_PATH, in_memory: bool = False):
+    def __init__(self, file_path: str = Constant.SQLITE_DATABASE_PATH, in_memory: bool = False, logger=None) -> None:
         """
             数据库连接，默认连接arcaea_database.db\ 
             接受：文件路径\ 
@@ -19,6 +18,8 @@ class Connect:
         """
         self.file_path = file_path
         self.in_memory: bool = in_memory
+        if logger is not None:
+            self.logger = logger
 
     def __enter__(self) -> sqlite3.Cursor:
         if self.in_memory:
@@ -37,7 +38,7 @@ class Connect:
             else:
                 self.conn.rollback()
 
-                current_app.logger.error(
+                self.logger.error(
                     traceback.format_exception(exc_type, exc_val, exc_tb))
 
         self.conn.commit()
@@ -49,9 +50,9 @@ class Connect:
 class Query:
     '''查询参数类'''
 
-    def __init__(self, query_able: list = None, quzzy_query_able: list = None, sort_able: list = None) -> None:
+    def __init__(self, query_able: list = None, fuzzy_query_able: list = None, sort_able: list = None) -> None:
         self.query_able: list = query_able  # None表示不限制
-        self.quzzy_query_able: list = quzzy_query_able  # None表示不限制
+        self.fuzzy_query_able: list = fuzzy_query_able  # None表示不限制
         self.sort_able: list = sort_able
 
         self.__limit: int = -1
@@ -115,7 +116,7 @@ class Query:
     def fuzzy_query_append(self, fuzzy_query: dict) -> None:
         if not isinstance(fuzzy_query, dict):
             raise InputError(api_error_code=-101)
-        if self.quzzy_query_able is not None and fuzzy_query and not set(fuzzy_query).issubset(set(self.quzzy_query_able)):
+        if self.fuzzy_query_able is not None and fuzzy_query and not set(fuzzy_query).issubset(set(self.fuzzy_query_able)):
             raise InputError(api_error_code=-102)
         if not self.__fuzzy_query:
             self.__fuzzy_query = fuzzy_query
@@ -216,7 +217,7 @@ class Sql:
         return ('insert into ' if insert_type is None else 'insert or ' + insert_type + ' into ') + table_name + ('(' + ','.join(key) + ')' if key else '') + ' values(' + ','.join(['?'] * (len(key) if value_len is None else value_len)) + ')'
 
     @staticmethod
-    def get_update_sql(table_name: str, d: dict = {}, query: 'Query' = None) -> str:
+    def get_update_sql(table_name: str, d: dict = None, query: 'Query' = None):
         if not d:
             return None
         sql_list = []
@@ -245,6 +246,13 @@ class Sql:
 
         return sql, sql_list
 
+    @staticmethod
+    def get_update_many_sql(table_name: str, key: list = None, where_key: list = None) -> str:
+        '''拼接update语句，这里不用Query类，也不用字典，请注意只返回sql语句'''
+        if not key or not where_key:
+            return None
+        return f"update {table_name} set {','.join([f'{k}=?' for k in key])} where {' and '.join([f'{k}=?' for k in where_key])}"
+
     @staticmethod
     def get_delete_sql(table_name: str, query: 'Query' = None):
         '''拼接删除语句，query中只有query和fuzzy_query会被处理'''
@@ -304,6 +312,13 @@ class Sql:
         sql, sql_list = self.get_update_sql(table_name, d, query)
         self.c.execute(sql, sql_list)
 
+    def update_many(self, table_name: str, key: list, value_list: list, where_key: list, where_value_list: list) -> None:
+        '''单表内行update多句sql语句，这里不用Query类，也不用字典，要求值list长度一致，有点像insert_many'''
+        if not key or not value_list or not where_key or not where_value_list or not len(key) == len(value_list[0]) or not len(where_key) == len(where_value_list[0]) or not len(value_list) == len(where_value_list):
+            raise ValueError
+        self.c.executemany(self.get_update_many_sql(
+            table_name, key, where_key), [x + y for x, y in zip(value_list, where_value_list)])
+
     def delete(self, table_name: str, query: 'Query' = None) -> None:
         '''删除，query中只有query和fuzzy_query会被处理'''
         sql, sql_list = self.get_delete_sql(table_name, query)

latest version/database/init/arc_data.py

@@ -1,45 +1,45 @@
 class InitData:
     char = ['hikari', 'tairitsu', 'kou', 'sapphire', 'lethe', 'hikari&tairitsu(reunion)', 'Tairitsu(Axium)', 'Tairitsu(Grievous Lady)', 'stella', 'Hikari & Fisica', 'ilith', 'eto', 'luna', 'shirabe', 'Hikari(Zero)', 'Hikari(Fracture)', 'Hikari(Summer)', 'Tairitsu(Summer)', 'Tairitsu & Trin',
-            'ayu', 'Eto & Luna', 'yume', 'Seine & Hikari', 'saya', 'Tairitsu & Chuni Penguin', 'Chuni Penguin', 'haruna', 'nono', 'MTA-XXX', 'MDA-21', 'kanae', 'Hikari(Fantasia)', 'Tairitsu(Sonata)', 'sia', 'DORO*C', 'Tairitsu(Tempest)', 'brillante', 'Ilith(Summer)', 'etude', 'Alice & Tenniel', 'Luna & Mia', 'areus', 'seele', 'isabelle', 'mir', 'lagrange', 'linka', 'nami', 'Saya & Elizabeth', 'lily', 'kanae(midsummer)', 'alice&tenniel(minuet)', 'tairitsu(elegy)', 'marija', 'vita', 'hikari(fatalis)', 'saki', 'setsuna', 'amane']
+            'ayu', 'Eto & Luna', 'yume', 'Seine & Hikari', 'saya', 'Tairitsu & Chuni Penguin', 'Chuni Penguin', 'haruna', 'nono', 'MTA-XXX', 'MDA-21', 'kanae', 'Hikari(Fantasia)', 'Tairitsu(Sonata)', 'sia', 'DORO*C', 'Tairitsu(Tempest)', 'brillante', 'Ilith(Summer)', 'etude', 'Alice & Tenniel', 'Luna & Mia', 'areus', 'seele', 'isabelle', 'mir', 'lagrange', 'linka', 'nami', 'Saya & Elizabeth', 'lily', 'kanae(midsummer)', 'alice&tenniel(minuet)', 'tairitsu(elegy)', 'marija', 'vita', 'hikari(fatalis)', 'saki', 'setsuna', 'amane', 'kou(winter)']
 
     skill_id = ['gauge_easy', '', '', '', 'note_mirror', 'skill_reunion', '', 'gauge_hard', 'frag_plus_10_pack_stellights', 'gauge_easy|frag_plus_15_pst&prs', 'gauge_hard|fail_frag_minus_100', 'frag_plus_5_side_light', 'visual_hide_hp', 'frag_plus_5_side_conflict', 'challenge_fullcombo_0gauge', 'gauge_overflow', 'gauge_easy|note_mirror', 'note_mirror', 'visual_tomato_pack_tonesphere',
-                'frag_rng_ayu', 'gaugestart_30|gaugegain_70', 'combo_100-frag_1', 'audio_gcemptyhit_pack_groovecoaster', 'gauge_saya', 'gauge_chuni', 'kantandeshou', 'gauge_haruna', 'frags_nono', 'gauge_pandora', 'gauge_regulus', 'omatsuri_daynight', '', '', 'sometimes(note_mirror|frag_plus_5)', 'scoreclear_aa|visual_scoregauge', 'gauge_tempest', 'gauge_hard', 'gauge_ilith_summer', '', 'note_mirror|visual_hide_far', 'frags_ongeki', 'gauge_areus', 'gauge_seele', 'gauge_isabelle', 'gauge_exhaustion', 'skill_lagrange', 'gauge_safe_10', 'frags_nami', 'skill_elizabeth', 'skill_lily', 'skill_kanae_midsummer', '', '', 'visual_ghost_skynotes', 'skill_vita', 'skill_fatalis', 'frags_ongeki_slash', 'frags_ongeki_hard', 'skill_amane']
+                'frag_rng_ayu', 'gaugestart_30|gaugegain_70', 'combo_100-frag_1', 'audio_gcemptyhit_pack_groovecoaster', 'gauge_saya', 'gauge_chuni', 'kantandeshou', 'gauge_haruna', 'frags_nono', 'gauge_pandora', 'gauge_regulus', 'omatsuri_daynight', '', '', 'sometimes(note_mirror|frag_plus_5)', 'scoreclear_aa|visual_scoregauge', 'gauge_tempest', 'gauge_hard', 'gauge_ilith_summer', '', 'note_mirror|visual_hide_far', 'frags_ongeki', 'gauge_areus', 'gauge_seele', 'gauge_isabelle', 'gauge_exhaustion', 'skill_lagrange', 'gauge_safe_10', 'frags_nami', 'skill_elizabeth', 'skill_lily', 'skill_kanae_midsummer', '', '', 'visual_ghost_skynotes', 'skill_vita', 'skill_fatalis', 'frags_ongeki_slash', 'frags_ongeki_hard', 'skill_amane', 'skill_kou_winter']
 
     skill_id_uncap = ['', '', 'frags_kou', '', 'visual_ink', '', '', '', '', '', '', 'eto_uncap', 'luna_uncap', 'shirabe_entry_fee',
-                      '', '', '', '', '', 'ayu_uncap', '', 'frags_yume', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
+                      '', '', '', '', '', 'ayu_uncap', '', 'frags_yume', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
 
     skill_unlock_level = [0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0,
-                          0, 0, 0, 8, 0, 14, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 8, 0, 0]
+                          0, 0, 0, 8, 0, 14, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 8, 0, 0, 0]
 
     frag1 = [55, 55, 60, 50, 47, 79, 47, 57, 41, 22, 50, 54, 60, 56, 78, 42, 41, 61, 52, 50, 52, 32,
-             42, 55, 45, 58, 43, 0.5, 68, 50, 62, 45, 45, 52, 44, 27, 59, 0, 45, 50, 50, 47, 47, 61, 43, 42, 38, 25, 58, 50, 61, 45, 45, 38, 34, 27, 18, 56, 47]
+             42, 55, 45, 58, 43, 0.5, 68, 50, 62, 45, 45, 52, 44, 27, 59, 0, 45, 50, 50, 47, 47, 61, 43, 42, 38, 25, 58, 50, 61, 45, 45, 38, 34, 27, 18, 56, 47, 30]
 
     prog1 = [35, 55, 47, 50, 60, 70, 60, 70, 58, 45, 70, 45, 42, 46, 61, 67, 49, 44, 28, 45, 24, 46, 52,
-             59, 62, 33, 58, 25, 63, 69, 50, 45, 45, 51, 34, 70, 62, 70, 45, 32, 32, 61, 47, 47, 37, 42, 50, 50, 45, 41, 61, 45, 45, 58, 50, 130, 18, 57, 55]
+             59, 62, 33, 58, 25, 63, 69, 50, 45, 45, 51, 34, 70, 62, 70, 45, 32, 32, 61, 47, 47, 37, 42, 50, 50, 45, 41, 61, 45, 45, 58, 50, 130, 18, 57, 55, 50]
 
     overdrive1 = [35, 55, 25, 50, 47, 70, 72, 57, 41, 7, 10, 32, 65, 31, 61, 53, 31, 47, 38, 12, 39, 18,
-                  48, 65, 45, 55, 44, 25, 46, 44, 33, 45, 45, 37, 25, 27, 50, 20, 45, 63, 21, 47, 61, 47, 65, 80, 38, 30, 49, 15, 34, 45, 45, 38, 67, 120, 44, 33, 55]
+                  48, 65, 45, 55, 44, 25, 46, 44, 33, 45, 45, 37, 25, 27, 50, 20, 45, 63, 21, 47, 61, 47, 65, 80, 38, 30, 49, 15, 34, 45, 45, 38, 67, 120, 44, 33, 55, 50]
 
     frag20 = [78, 80, 90, 75, 70, 79, 70, 79, 65, 40, 50, 80, 90, 82, 0, 61, 67, 92, 85, 50, 86, 52,
-              65, 85, 67, 88, 64, 0.5, 95, 70, 95, 50, 80, 87, 71, 50, 85, 0, 80, 75, 50, 70, 70, 90, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 35, 85, 47]
+              65, 85, 67, 88, 64, 0.5, 95, 70, 95, 50, 80, 87, 71, 50, 85, 0, 80, 75, 50, 70, 70, 90, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 35, 85, 47, 50]
 
     prog20 = [61, 80, 70, 75, 90, 70, 90, 102, 84, 78, 105, 67, 63, 68, 0, 99, 80, 66, 46, 83, 40, 73,
-              80, 90, 93, 50, 86, 78, 89, 98, 75, 80, 50, 64, 55, 100, 90, 110, 80, 50, 74, 90, 70, 70, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 35, 86, 92]
+              80, 90, 93, 50, 86, 78, 89, 98, 75, 80, 50, 64, 55, 100, 90, 110, 80, 50, 74, 90, 70, 70, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 35, 86, 92, 80]
 
     overdrive20 = [61, 80, 47, 75, 70, 70, 95, 79, 65, 31, 50, 59, 90, 58, 0, 78, 50, 70, 62, 49, 64,
-                   46, 73, 95, 67, 84, 70, 78, 69, 70, 50, 80, 80, 63, 25, 50, 72, 55, 50, 95, 55, 70, 90, 70, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 85, 50, 92]
+                   46, 73, 95, 67, 84, 70, 78, 69, 70, 50, 80, 80, 63, 25, 50, 72, 55, 50, 95, 55, 70, 90, 70, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 85, 50, 92, 50]
 
     frag30 = [88, 90, 100, 75, 80, 89, 70, 79, 65, 40, 50, 90, 100, 92, 0, 61, 67, 92, 85, 50, 86, 62,
-              65, 85, 67, 88, 74, 0.5, 105, 80, 95, 50, 80, 87, 71, 50, 95, 0, 80, 75, 50, 70, 80, 100, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 35, 85, 47]
+              65, 85, 67, 88, 74, 0.5, 105, 80, 95, 50, 80, 87, 71, 50, 95, 0, 80, 75, 50, 70, 80, 100, 65, 80, 61, 50, 68, 60, 90, 67, 50, 60, 51, 50, 35, 85, 47, 50]
 
     prog30 = [71, 90, 80, 75, 100, 80, 90, 102, 84, 78, 105, 77, 73, 78, 0, 99, 80, 66, 46, 93, 40, 83,
-              80, 90, 93, 50, 96, 88, 99, 108, 75, 80, 50, 64, 55, 100, 100, 110, 80, 50, 74, 90, 80, 80, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 35, 86, 92]
+              80, 90, 93, 50, 96, 88, 99, 108, 75, 80, 50, 64, 55, 100, 100, 110, 80, 50, 74, 90, 80, 80, 56, 80, 79, 55, 65, 59, 90, 50, 90, 90, 75, 210, 35, 86, 92, 80]
 
     overdrive30 = [71, 90, 57, 75, 80, 80, 95, 79, 65, 31, 50, 69, 100, 68, 0, 78, 50, 70, 62, 59, 64,
-                   56, 73, 95, 67, 84, 80, 88, 79, 80, 50, 80, 80, 63, 25, 50, 82, 55, 50, 95, 55, 70, 100, 80, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 85, 50, 92]
+                   56, 73, 95, 67, 84, 80, 88, 79, 80, 50, 80, 80, 63, 25, 50, 82, 55, 50, 95, 55, 70, 100, 80, 99, 80, 61, 40, 69, 62, 51, 90, 67, 60, 100, 200, 85, 50, 92, 50]
 
     char_type = [1, 0, 0, 0, 0, 0, 0, 2, 0, 1, 2, 0, 0, 0, 2, 3, 1, 0, 0, 0, 1,
-                 0, 0, 0, 0, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 2, 2, 2, 0, 0, 0, 2, 2, 2, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 2, 3, 0, 2, 2]
+                 0, 0, 0, 0, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 2, 2, 2, 0, 0, 0, 2, 2, 2, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 2, 3, 0, 2, 2, 0]
 
     char_core = {
         0: [{'core_id': 'core_hollow', 'amount': 25}, {'core_id': 'core_desolate', 'amount': 5}],
@@ -64,7 +64,7 @@ class InitData:
              'core_ambivalent', 'core_scarlet', 'core_groove', 'core_generic', 'core_binary', 'core_colorful', 'core_course_skip_purchase']
 
     world_songs = ["babaroque", "shadesoflight", "kanagawa", "lucifer", "anokumene", "ignotus", "rabbitintheblackroom", "qualia", "redandblue", "bookmaker", "darakunosono", "espebranch", "blacklotus", "givemeanightmare", "vividtheory", "onefr", "gekka", "vexaria3", "infinityheaven3", "fairytale3", "goodtek3", "suomi", "rugie", "faintlight", "harutopia", "goodtek", "dreaminattraction", "syro", "diode", "freefall", "grimheart", "blaster",
-                   "cyberneciacatharsis", "monochromeprincess", "revixy", "vector", "supernova", "nhelv", "purgatorium3", "dement3", "crossover", "guardina", "axiumcrisis", "worldvanquisher", "sheriruth", "pragmatism", "gloryroad", "etherstrike", "corpssansorganes", "lostdesire", "blrink", "essenceoftwilight", "lapis", "solitarydream", "lumia3", "purpleverse", "moonheart3", "glow", "enchantedlove", "take", "lifeispiano", "vandalism", "nexttoyou3", "lostcivilization3", "turbocharger", "bookmaker3", "laqryma3", "kyogenkigo", "hivemind", "seclusion", "quonwacca3", "bluecomet", "energysynergymatrix", "gengaozo", "lastendconductor3", "antithese3", "qualia3", "kanagawa3", "heavensdoor3", "pragmatism3", "nulctrl", "avril", "ddd", "merlin3", "omakeno3", "nekonote", "sanskia", 'altair', 'mukishitsu', 'trapcrow', 'redandblue3', 'ignotus3', 'singularity3', 'dropdead3', 'arcahv', 'freefall3']
+                   "cyberneciacatharsis", "monochromeprincess", "revixy", "vector", "supernova", "nhelv", "purgatorium3", "dement3", "crossover", "guardina", "axiumcrisis", "worldvanquisher", "sheriruth", "pragmatism", "gloryroad", "etherstrike", "corpssansorganes", "lostdesire", "blrink", "essenceoftwilight", "lapis", "solitarydream", "lumia3", "purpleverse", "moonheart3", "glow", "enchantedlove", "take", "lifeispiano", "vandalism", "nexttoyou3", "lostcivilization3", "turbocharger", "bookmaker3", "laqryma3", "kyogenkigo", "hivemind", "seclusion", "quonwacca3", "bluecomet", "energysynergymatrix", "gengaozo", "lastendconductor3", "antithese3", "qualia3", "kanagawa3", "heavensdoor3", "pragmatism3", "nulctrl", "avril", "ddd", "merlin3", "omakeno3", "nekonote", "sanskia", 'altair', 'mukishitsu', 'trapcrow', 'redandblue3', 'ignotus3', 'singularity3', 'dropdead3', 'arcahv', 'freefall3', 'partyvinyl3']
 
     world_unlocks = ["scenery_chap1", "scenery_chap2",
                      "scenery_chap3", "scenery_chap4", "scenery_chap5", "scenery_chap6", "scenery_chap7"]
@@ -75,12 +75,12 @@ class InitData:
     role_caption = ['系统', '管理员', '用户', '查询接口']
 
     power = ['system', 'select', 'select_me', 'change', 'change_me', 'grant',
-             'grant_inf', 'select_song_rank', 'select_song_info']
+             'grant_inf', 'select_song_rank', 'select_song_info', 'select_song_rank_top']
     power_caption = ['系统权限', '总体查询权限', '自我查询权限', '总体修改权限',
-                     '自我修改权限', '授权权限', '下级授权权限', '歌曲排行榜查询权限', '歌曲信息查询权限']
+                     '自我修改权限', '授权权限', '下级授权权限', '歌曲排行榜查询权限', '歌曲信息查询权限', '歌曲排行榜有限查询权限']
 
     role_power = {'system': power,
-                  'admin': ['select', 'select_me', 'change_me', 'grant_inf', 'select_song_rank', 'select_song_info'],
+                  'admin': ['select', 'select_me', 'change_me', 'grant_inf', 'select_song_rank_top', 'select_song_info'],
                   'user': ['select_me', 'change_me', 'select_song_rank', 'select_song_info'],
                   'selector': ['select']
                   }

latest version/database/init/log_tables.sql

@@ -0,0 +1,25 @@
+create table if not exists cache(key text primary key,
+value text,
+expire_time int
+);
+create table if not exists user_score(user_id int,
+song_id text,
+difficulty int,
+time_played int,
+score int,
+shiny_perfect_count int,
+perfect_count int,
+near_count int,
+miss_count int,
+health int,
+modifier int,
+clear_type int,
+rating real,
+primary key(user_id, song_id, difficulty, time_played)
+);
+
+create index if not exists user_score_1 on user_score (song_id, difficulty);
+create index if not exists user_score_2 on user_score (time_played);
+
+PRAGMA journal_mode = WAL;
+PRAGMA default_cache_size = 4000;

latest version/database/init/singles.json

@@ -1252,5 +1252,23 @@
 		],
 		"orig_price": 100,
 		"price": 100
+	},
+	{
+		"name": "capella",
+		"items": [
+			{
+				"type": "single",
+				"id": "capella",
+				"is_available": true
+			},
+			{
+				"type": "core",
+				"amount": 1,
+				"id": "core_generic",
+				"is_available": true
+			}
+		],
+		"orig_price": 100,
+		"price": 100
 	}
 ]

latest version/main.py

@@ -27,7 +27,7 @@ import server
 import web.index
 import web.login
 from core.constant import Constant
-from core.download import (UserDownload, initialize_songfile)
+from core.download import UserDownload
 from core.error import ArcError, NoAccess, RateLimit
 from core.init import FileChecker
 from core.sql import Connect
@@ -128,6 +128,18 @@ def tcp_server_run():
             app.run(Config.HOST, Config.PORT)
 
 
+def generate_log_file_dict(level: str, filename: str) -> dict:
+    return {
+        "class": "logging.handlers.RotatingFileHandler",
+        "maxBytes": 1024 * 1024,
+        "backupCount": 1,
+        "encoding": "utf-8",
+        "level": level,
+        "formatter": "default",
+        "filename": filename
+    }
+
+
 def main():
     log_dict = {
         'version': 1,
@@ -141,15 +153,7 @@ def main():
                 'stream': 'ext://flask.logging.wsgi_errors_stream',
                 'formatter': 'default'
             },
-            "error_file": {
-                "class": "logging.handlers.RotatingFileHandler",
-                "maxBytes": 1024 * 1024,
-                "backupCount": 1,
-                "encoding": "utf-8",
-                "level": "ERROR",
-                "formatter": "default",
-                "filename": "./log/error.log"
-            }
+            "error_file": generate_log_file_dict('ERROR', './log/error.log')
         },
         'formatters': {
             'default': {
@@ -159,50 +163,30 @@ def main():
     }
     if Config.ALLOW_INFO_LOG:
         log_dict['root']['handlers'].append('info_file')
-        log_dict['handlers']['info_file'] = {
-            "class": "logging.handlers.RotatingFileHandler",
-            "maxBytes": 1024 * 1024,
-            "backupCount": 1,
-            "encoding": "utf-8",
-            "level": "INFO",
-            "formatter": "default",
-            "filename": "./log/info.log"
-        }
+        log_dict['handlers']['info_file'] = generate_log_file_dict(
+            'INFO', './log/info.log')
     if Config.ALLOW_WARNING_LOG:
         log_dict['root']['handlers'].append('warning_file')
-        log_dict['handlers']['warning_file'] = {
-            "class": "logging.handlers.RotatingFileHandler",
-            "maxBytes": 1024 * 1024,
-            "backupCount": 1,
-            "encoding": "utf-8",
-            "level": "WARNING",
-            "formatter": "default",
-            "filename": "./log/warning.log"
-        }
+        log_dict['handlers']['warning_file'] = generate_log_file_dict(
+            'WARNING', './log/warning.log')
 
     dictConfig(log_dict)
 
-    if not FileChecker(app).check_before_run():
-        app.logger.error('Something wrong. The server will not run.')
+    Connect.logger = app.logger
+    if not FileChecker(app.logger).check_before_run():
+        app.logger.error('Some errors occurred. The server will not run.')
         input('Press ENTER key to exit.')
         sys.exit()
 
-    app.logger.info("Start to initialize song data...")
-    try:
-        initialize_songfile()
-        app.logger.info('Complete!')
-    except:
-        app.logger.warning('Initialization error!')
-
     if Config.LINKPLAY_HOST and Config.SET_LINKPLAY_SERVER_AS_SUB_PROCESS:
         from linkplay_server import link_play
         process = [Process(target=link_play, args=(
             Config.LINKPLAY_HOST, int(Config.LINKPLAY_UDP_PORT), int(Config.LINKPLAY_TCP_PORT)))]
         [p.start() for p in process]
-        app.logger.info("Link Play UDP server is running on " +
-                        Config.LINKPLAY_HOST + ':' + str(Config.LINKPLAY_UDP_PORT) + " ...")
-        app.logger.info("Link Play TCP server is running on " +
-                        Config.LINKPLAY_HOST + ':' + str(Config.LINKPLAY_TCP_PORT) + " ...")
+        app.logger.info(
+            f"Link Play UDP server is running on {Config.LINKPLAY_HOST}:{Config.LINKPLAY_UDP_PORT} ...")
+        app.logger.info(
+            f"Link Play TCP server is running on {Config.LINKPLAY_HOST}:{Config.LINKPLAY_TCP_PORT} ...")
         tcp_server_run()
         [p.join() for p in process]
     else:

latest version/run.bat

@@ -1,3 +1,4 @@
 cd /d %~dp0
 :: Develop server
-python -B main.py
+python -B main.py
+pause

latest version/server/arcscore.py

@@ -1,56 +0,0 @@
-from core.sql import Connect
-
-
-def calculate_rating(defnum, score):
-    # 计算rating
-    if score >= 10000000:
-        ptt = defnum + 2
-    elif score < 9800000:
-        ptt = defnum + (score-9500000) / 300000
-        if ptt < 0 and defnum != -10:
-            ptt = 0
-    else:
-        ptt = defnum + 1 + (score-9800000) / 200000
-
-    return ptt
-
-
-def refresh_all_score_rating():
-    # 刷新所有best成绩的rating
-    error = 'Unknown error.'
-
-    with Connect() as c:
-        c.execute(
-            '''select song_id, rating_pst, rating_prs, rating_ftr, rating_byn from chart''')
-        x = c.fetchall()
-
-    if x:
-        song_list = [i[0] for i in x]
-        with Connect() as c:
-            c.execute('''update best_score set rating=0 where song_id not in ({0})'''.format(
-                ','.join(['?']*len(song_list))), tuple(song_list))
-            for i in x:
-                for j in range(0, 4):
-                    defnum = -10  # 没在库里的全部当做定数-10
-                    if i is not None:
-                        defnum = float(i[j+1]) / 10
-                        if defnum <= 0:
-                            defnum = -10  # 缺少难度的当做定数-10
-
-                    c.execute('''select user_id, score from best_score where song_id=:a and difficulty=:b''', {
-                              'a': i[0], 'b': j})
-                    y = c.fetchall()
-                    if y:
-                        for k in y:
-                            ptt = calculate_rating(defnum, k[1])
-                            if ptt < 0:
-                                ptt = 0
-
-                            c.execute('''update best_score set rating=:a where user_id=:b and song_id=:c and difficulty=:d''', {
-                                      'a': ptt, 'b': k[0], 'c': i[0], 'd': j})
-            error = None
-
-    else:
-        error = 'No song data.'
-
-    return error

latest version/server/auth.py

@@ -5,7 +5,7 @@ from core.config_manager import Config
 from core.error import ArcError, NoAccess
 from core.sql import Connect
 from core.user import UserAuth, UserLogin
-from flask import Blueprint, jsonify, request
+from flask import Blueprint, g, jsonify, request
 
 from .func import arc_try, error_return
 
@@ -55,6 +55,7 @@ def auth_required(request):
                     user = UserAuth(c)
                     user.token = headers['Authorization'][7:]
                     user_id = user.token_get_id()
+                    g.user = user
                 except ArcError as e:
                     return error_return(e)
             return view(user_id, *args, **kwargs)

latest version/server/func.py

@@ -3,7 +3,7 @@ from traceback import format_exc
 
 from core.config_manager import Config
 from core.error import ArcError
-from flask import current_app, jsonify
+from flask import current_app, g, jsonify
 
 default_error = ArcError('Unknown Error', status=500)
 
@@ -83,6 +83,9 @@ def arc_try(view):
         except ArcError as e:
             if Config.ALLOW_WARNING_LOG:
                 current_app.logger.warning(format_exc())
+            user = g.get("user", None)
+            current_app.logger.warning(
+                f'{user.user_id if user is not None else ""} - {e.error_code}|{e.api_error_code}: {e}')
             return error_return(e)
 
     return wrapped_view

latest version/server/init.py

@@ -1,91 +0,0 @@
-import os
-from shutil import copy, copy2
-
-from core.sql import Connect
-from database.database_initialize import ARCAEA_SERVER_VERSION, main
-from web.system import update_database
-
-
-def try_rename(path, new_path):
-    # 尝试重命名文件，并尝试避免命名冲突，返回最终路径
-    final_path = new_path
-    if os.path.exists(new_path):
-        i = 1
-        while os.path.exists(new_path + str(i)):
-            i += 1
-        os.rename(path, new_path + str(i))
-        final_path = new_path + str(i)
-    else:
-        os.rename(path, new_path)
-
-    return final_path
-
-
-def check_before_run(app):
-    # 运行前检查关键文件，返回布尔值，其实是因为有人经常忘了
-
-    f = True
-
-    if not os.path.exists('database'):
-        app.logger.warning('Folder `database` is missing.')
-        f = False
-
-    if not os.path.exists('database/songs'):
-        app.logger.warning('Folder `database/songs` is missing.')
-        f = False
-
-    if not os.path.exists('database/arcaea_database.db'):
-        app.logger.warning('File `database/arcaea_database.db` is missing.')
-        f = False
-        try:
-            app.logger.info(
-                'Try to new the file `database/arcaea_database.db`.')
-            main('./database/')
-            app.logger.info(
-                'Success to new the file `database/arcaea_database.db`.')
-            f = True
-        except:
-            app.logger.warning(
-                'Fail to new the file `database/arcaea_database.db`.')
-
-    else:
-        with Connect() as c:
-            try:
-                c.execute('''select value from config where id="version"''')
-                x = c.fetchone()
-            except:
-                x = None
-
-        # 数据库自动更新，不强求
-        if not x or x[0] != ARCAEA_SERVER_VERSION:
-            app.logger.warning(
-                'Maybe the file `database/arcaea_database.db` is an old version.')
-            try:
-                app.logger.info(
-                    'Try to update the file `database/arcaea_database.db`.')
-
-                path = try_rename('database/arcaea_database.db',
-                                  'database/arcaea_database.db.bak')
-
-                try:
-                    copy2(path, 'database/arcaea_database.db')
-                except:
-                    copy(path, 'database/arcaea_database.db')
-
-                if os.path.isfile("database/old_arcaea_database.db"):
-                    os.remove('database/old_arcaea_database.db')
-
-                try_rename('database/arcaea_database.db',
-                           'database/old_arcaea_database.db')
-
-                main('./database/')
-                update_database()
-
-                app.logger.info(
-                    'Success to update the file `database/arcaea_database.db`.')
-
-            except:
-                app.logger.warning(
-                    'Fail to update the file `database/arcaea_database.db`.')
-
-    return f

latest version/server/present.py

@@ -1,11 +1,10 @@
-from core.error import ArcError
 from core.present import UserPresent, UserPresentList
 from core.sql import Connect
 from core.user import UserOnline
 from flask import Blueprint, request
 
 from .auth import auth_required
-from .func import arc_try, error_return, success_return
+from .func import arc_try, success_return
 
 bp = Blueprint('present', __name__, url_prefix='/present')
 

latest version/web/index.py

@@ -1,9 +1,8 @@
 import os
 import time
 
-import server.arcscore
-from core.download import DownloadList, initialize_songfile
 from core.init import FileChecker
+from core.operation import RefreshAllScoreRating, RefreshSongFileCache
 from core.rank import RankList
 from core.sql import Connect
 from flask import Blueprint, flash, redirect, render_template, request, url_for
@@ -290,8 +289,7 @@ def update_database():
 def update_song_hash():
     # 更新数据库内谱面文件hash值
     try:
-        DownloadList.clear_all_cache()
-        initialize_songfile()
+        RefreshSongFileCache().run()
         flash('数据刷新成功 Success refresh data.')
     except:
         flash('Something error!')
@@ -302,11 +300,8 @@ def update_song_hash():
 @login_required
 def update_song_rating():
     # 更新所有分数的rating
-    error = server.arcscore.refresh_all_score_rating()
-    if error:
-        flash(error)
-    else:
-        flash('数据刷新成功 Success refresh data.')
+    RefreshAllScoreRating().run()
+    flash('数据刷新成功 Success refresh data.')
     return render_template('web/updatedatabase.html')