Compare commits

...

13 Commits

Author SHA1 Message Date
ad82d6f692 261117_01_FIX: 순위 문항 import 시 {T} 기타 보기의 STRUCTURE 텍스트 변수 누락 수정 2026-11-17 16:36:53 +09:00
55d068a17f 261111_02_ENH: Kiwi 사전·치환 등록 경고 로그의 인자 계산을 변수로 분리 2026-11-11 12:29:38 +09:00
6e1d57414e 261111_01_ENH: Kiwi 형태소 분석 정확도 개선 및 품사 합산·복합명사·치환·불용어 옵션 추가 2026-11-11 10:37:13 +09:00
fb76c9dcb3 261104_04_HOT: 엑셀파일 생성 시 스트리밍 방식으로 생성 2026-11-04 18:31:44 +09:00
7eb4d3bc9e 261104_03_HOT: 트랜젝션 누수 관련 긴급 조치 2 2026-11-04 18:26:40 +09:00
47c598a61c 261104_02_HOT: 트랜젝션 누수 관련 긴급 조치 2026-11-04 17:18:38 +09:00
215a5059e3 261104_01_ENH: 기존 통계 결과를 유지하며 데이터 접근 및 문항·쿼터 설정 해석 분리 2026-11-04 15:05:08 +09:00
9fa4070be2 261103_01_ENH: 현재 인증 사용자 정보 조회 API 추가 및 응답 캐시 방지 적용 2026-11-03 14:09:47 +09:00
32b3d374a3 261102_01_ENH: 프로젝트 접근 권한 확인 API 추가 및 공통 사용자 Context 적용 2026-11-02 15:12:58 +09:00
9bf9cbe135 261028_01_ENH: prod의 share base url 변경 2026-10-28 08:32:49 +09:00
0e78680571 261027_03_ENH: 파생 변수 데이터 개별 생성시 Status=10 조건을 뺌 2026-10-27 14:43:48 +09:00
4d4406ed2b 261027_02_ENH: url 생성에 대한 도메인 분리 2026-10-27 12:21:42 +09:00
79c9619733 261027_01_ENH: 데이터 집계 및 통계 처리 흐름도 추가 2026-10-27 09:07:16 +09:00
58 changed files with 1720 additions and 383 deletions

View File

@@ -21,6 +21,7 @@ Source: .agent/consumer-references/platform-capabilities.md
- Common Web may register error handling, tracing/MDC, request-user resolution, enum conversion, and CORS behavior. Check for overlap before defining consumer-local equivalents. - Common Web may register error handling, tracing/MDC, request-user resolution, enum conversion, and CORS behavior. Check for overlap before defining consumer-local equivalents.
- Swagger support requires springdoc and defines the shared Bearer/API-key schemes and grouped paths. - Swagger support requires springdoc and defines the shared Bearer/API-key schemes and grouped paths.
- When Swagger auto-configuration is active (`kri.swagger.enabled=true`, default), `RequestUserContext` controller arguments are ignored as request inputs globally, including their `authenticated`, `clientIp`, and `userRole` properties. Runtime argument resolution is unchanged. - When Swagger auto-configuration is active (`kri.swagger.enabled=true`, default), `RequestUserContext` controller arguments are ignored as request inputs globally, including their `authenticated`, `clientIp`, and `userRole` properties. Runtime argument resolution is unchanged.
- `RequestUserContextProvider` is a public injectable API for resolving the same request identity and client IP used by controller arguments. Its default registration requires Web integration and JwtService; a consumer provider overrides it. Existing principal/details/verified-Bearer fallback, empty-context behavior, resolver overrides and the JwtService resolver constructor remain unchanged. The provider does not enforce authentication or business authorization.
- Redis and scheduling are opt-in through their `kri.*.enabled` settings and matching runtime dependencies. - Redis and scheduling are opt-in through their `kri.*.enabled` settings and matching runtime dependencies.
- HTTP bypass targets are server-configured. Never accept an arbitrary full target URL from a frontend; retain allowed-path checks and do not forward hop-by-hop headers. - HTTP bypass targets are server-configured. Never accept an arbitrary full target URL from a frontend; retain allowed-path checks and do not forward hop-by-hop headers.
- The external API client returns response bodies as documented, while bypass responses preserve raw bytes and target `Content-Type`. - The external API client returns response bodies as documented, while bypass responses preserve raw bytes and target `Content-Type`.

View File

@@ -1,6 +1,6 @@
## KART Server Repository Rules ## KART Server Repository Rules
- Java 17, Spring Boot 3.5.7, KriCommon `com.kri.common:starter:beta.1.6.9`. - Java 17, Spring Boot 3.5.7, KriCommon `com.kri.common:starter:beta.1.7.1`.
- Use MyBatis with explicit KON, KONKIWI and kart (Report DB) SQL Server sessions and transaction managers. Do not infer a source from pkey alone. - Use MyBatis with explicit KON, KONKIWI and kart (Report DB) SQL Server sessions and transaction managers. Do not infer a source from pkey alone.
- ProjectType codes are 0=KON and 1=KIWI; reject unsupported types and legacy key markers in new project identifiers. - ProjectType codes are 0=KON and 1=KIWI; reject unsupported types and legacy key markers in new project identifiers.
- This API is for the new system. Legacy API compatibility is not required; shared database compatibility is required. - This API is for the new system. Legacy API compatibility is not required; shared database compatibility is required.
@@ -30,6 +30,7 @@
- Follow Contact: inject com.kri.common.feature.web.RequestUserContext ctx into controller methods; use ctx.getUserRole() and ctx.getUserTeam() for authorization and pass them to the service as needed. - Follow Contact: inject com.kri.common.feature.web.RequestUserContext ctx into controller methods; use ctx.getUserRole() and ctx.getUserTeam() for authorization and pass them to the service as needed.
- Use only the context resolved by common authentication, never role/team fields submitted in request DTOs. - Use only the context resolved by common authentication, never role/team fields submitted in request DTOs.
- Place Spring MVC interceptors under the interceptor package. Inject KriCommon RequestUserContextProvider to resolve request identity; keep project and share-link authorization policies in KART.
- Do not introduce separate PrincipalKind/ProjectPermission/ReportPage enums or generic grant models for this purpose. - Do not introduce separate PrincipalKind/ProjectPermission/ReportPage enums or generic grant models for this purpose.
- External authentication must still enforce its stored project/page scope and restrict access to report reads and aggregate Excel; role/team alone must not authorize access outside the link scope. - External authentication must still enforce its stored project/page scope and restrict access to report reads and aggregate Excel; role/team alone must not authorize access outside the link scope.
@@ -111,7 +112,7 @@
- Follow existing question organization. Return N and percent together, including Total. Preserve repeated rank/matrix occurrences, original numeric missing/mean/statistic rules, scale/reverse/NPS, grouped values and TABULATION denominators. Frontend chooses PRTMODE presentation without requery. - Follow existing question organization. Return N and percent together, including Total. Preserve repeated rank/matrix occurrences, original numeric missing/mean/statistic rules, scale/reverse/NPS, grouped values and TABULATION denominators. Frontend chooses PRTMODE presentation without requery.
- Keep original empty-base semantics: ordinary cell/Total percentages are 0; undefined group/statistic values are null with '-' display. Use 15-significant-digit C# double-format behavior and midpoint-away rounding, validated by legacy fixtures. - Keep original empty-base semantics: ordinary cell/Total percentages are 0; undefined group/statistic values are null with '-' display. Use 15-significant-digit C# double-format behavior and midpoint-away rounding, validated by legacy fixtures.
- StatisticsMapper uses survey sessions, parameterized complete/TEST/date conditions, original distinct derived/profile joins and selected storage-group projection. Filter data is private and separate from projected aggregation data. AST filters never inject user SQL. - StatisticsMapper uses survey sessions, parameterized complete/TEST/date conditions, original distinct derived/profile joins and selected storage-group projection. Filter data is private and separate from projected aggregation data. AST filters never inject user SQL.
- User approved the latest Kiwi model after finding the original DLL was 0.15.2. Use matching official KiwiJava/model 0.24.0 CONG instead of pinning SBG. Model-result changes are allowed; preserve Report NLP dictionary/tag scopes, word scores 10/50/100, tag/length/SL selection and stable top-100 frequency logic. Add no new stopword rules. - User approved the latest Kiwi model after finding the original DLL was 0.15.2. Use matching official KiwiJava/model 0.24.0 CONG instead of pinning SBG. Model-result changes are allowed; preserve Report NLP dictionary/tag scopes, word scores 10/50/100, tag/length/SL selection and stable top-100 frequency logic. Stopwords exist only as the REPORT_NLP_WORD STOP rows described under Kiwi NLP accuracy; add no other stopword rules.
- Configure kri.kart.nlp.model-directory as ./nlp_model/v0.24.0, install model separately, ignore runtime models in Git, and choose the platform-specific official jar at build time. Do not claim Ubuntu execution from Windows tests. Missing native/model infrastructure is explicit 503, not empty successful NLP output. - Configure kri.kart.nlp.model-directory as ./nlp_model/v0.24.0, install model separately, ignore runtime models in Git, and choose the platform-specific official jar at build time. Do not claim Ubuntu execution from Windows tests. Missing native/model infrastructure is explicit 503, not empty successful NLP output.
- Per user decision, the legacy external-Referer condition is removed: every general statistics batch writes Get Report Data once through LogService. Quota results write QUOTA VIEW only, never Get Report Data. Do not add a separate logging endpoint. - Per user decision, the legacy external-Referer condition is removed: every general statistics batch writes Get Report Data once through LogService. Quota results write QUOTA VIEW only, never Get Report Data. Do not add a separate logging endpoint.
- Validate through original C# synthetic calculation fixtures, authenticated API/Swagger tests, source-bound MyBatis tests and opt-in actual native analysis with -PkiwiModelDirectory. Actual SQL Server equivalence and deployment performance remain separate checks. - Validate through original C# synthetic calculation fixtures, authenticated API/Swagger tests, source-bound MyBatis tests and opt-in actual native analysis with -PkiwiModelDirectory. Actual SQL Server equivalence and deployment performance remain separate checks.
@@ -168,7 +169,7 @@
- Contact calls the legacy scoped project-list endpoint and skey-to-share sequence; KON/KON-KIWI survey.js calls respondent-derived recalculation at END. Preserve new-server-only migration scope: do not modify those consumer repositories automatically. - Contact calls the legacy scoped project-list endpoint and skey-to-share sequence; KON/KON-KIWI survey.js calls respondent-derived recalculation at END. Preserve new-server-only migration scope: do not modify those consumer repositories automatically.
- GET /project/list without search keeps existing screen-list behavior. With projectType/intranetCode/userTeam, use the legacy linked query: first underscore code segment, R-team exact GKEY else exclude M101, keyword overrides code with pkey/title LIKE, newest first. Do not mix ordinary-screen year/must-view/status/company/din-owner filters or KIWI title prefix into linked results. Query userTeam is not an authentication claim; additionally enforce the authenticated read visibility rules below, without adding ordinary-screen business filters. - GET /project/list without search keeps existing screen-list behavior. With projectType/intranetCode/userTeam, use the legacy linked query: first underscore code segment, R-team exact GKEY else exclude M101, keyword overrides code with pkey/title LIKE, newest first. Do not mix ordinary-screen year/must-view/status/company/din-owner filters or KIWI title prefix into linked results. Query userTeam is not an authentication claim; additionally enforce the authenticated read visibility rules below, without adding ordinary-screen business filters.
- POST /project/{projectType}/{pkey}/derived/respondent/{respondentId}/recalculate reuses current calculation in definition order, ensures the legacy empty derived row, scopes group clear/query/save to PKEY+IDKEY, reads actual completed responses only and uses the selected survey transaction. Preserve historical date values on no match and add no new SiteLog point. - POST /project/{projectType}/{pkey}/derived/respondent/{respondentId}/recalculate reuses current calculation in definition order, ensures the legacy empty derived row, scopes group clear/query/save to PKEY+IDKEY, reads non-TEST responses regardless of Status (user decision 2026-10-27, allowing calculation before completion status is saved) and uses the selected survey transaction. Whole-project recalculation retains Status = 10. Preserve historical date values on no match and add no new SiteLog point.
- Local prefix search did not locate a PRJCT_/KRIC_SYS_ caller; do not claim production non-use or remove legacy registrations. Existing Common API Key auth covers new server-to-server calls; verified REPORT_URL covers CLIENT. Do not add unauthenticated system-name/prefix-to-JWT exchange or trust request-supplied identity. Incoming service-code kart-server API Key registration is separate from the outgoing Intranet key. - Local prefix search did not locate a PRJCT_/KRIC_SYS_ caller; do not claim production non-use or remove legacy registrations. Existing Common API Key auth covers new server-to-server calls; verified REPORT_URL covers CLIENT. Do not add unauthenticated system-name/prefix-to-JWT exchange or trust request-supplied identity. Incoming service-code kart-server API Key registration is separate from the outgoing Intranet key.
- Existing browser survey hooks need a server-side proxy/API Key integration at later cutover. Caller conversion, real API Key registration/traffic, shared DB checks, general authorization and deployment remain separate work. - Existing browser survey hooks need a server-side proxy/API Key integration at later cutover. Caller conversion, real API Key registration/traffic, shared DB checks, general authorization and deployment remain separate work.
@@ -186,3 +187,14 @@
- Apply team/assignment before candidate TOP 50 and ordering, never after Java merge/TOP 6. Must-view does not override authorization. Linked list retains its business query team and adds separately named authenticated-team/role/assignment constraints. - Apply team/assignment before candidate TOP 50 and ordering, never after Java merge/TOP 6. Must-view does not override authorization. Linked list retains its business query team and adds separately named authenticated-team/role/assignment constraints.
- Derived management list: role 1001 sees all project definitions; others see matching GKEY only. Keep normal questions/report metadata unfiltered by this management rule. - Derived management list: role 1001 sees all project definitions; others see matching GKEY only. Keep normal questions/report metadata unfiltered by this management rule.
- Scope is these read conditions only. Do not add generic per-project/mutation/owner authorization, change original aggregations/Excel, or weaken External scope. Validate actual bound mapper SQL and H2 results on both sources, empty assignments, precise id routing, trusted JWT/API Key context versus forged query fields; no shared DB execution. - Scope is these read conditions only. Do not add generic per-project/mutation/owner authorization, change original aggregations/Excel, or weaken External scope. Validate actual bound mapper SQL and H2 results on both sources, empty assignments, precise id routing, trusted JWT/API Key context versus forged query fields; no shared DB execution.
## Kiwi NLP accuracy (user decisions 2026-11-10)
- The legacy C# report services are retired; REPORT_NLP_WORD/TAG are no longer read by them.
- KiwiNlpService builds with BuildOption.default_ and analyzes with Match.allWithNormalizing | oovChrModel plus basicTypoSetWithContinual typo correction (threshold 2.5). Do not revert to AnalyzeOption(0).
- User dictionary: register each (text, tag) once with its highest scope score, because Kiwi keeps the last duplicate score and the query has no order. Trim/uppercase tags. Log only registered/skipped/rejected counts, never words.
- Statistics options: wordMergeMode 1=ALL, 2=NOUN (default; NNG/NNP/NNB/NR/NP/SL/SH), 3=NONE sums the same text across tags, keeping the most frequent tag (first on ties) and first-appearance order. compoundNounMaxTokens 1-5 (default 3; 1 disables) joins noun prefixes/suffixes and merges NNG/NNP/SL runs attached within one word when they contain a Korean noun; NNP if any part is NNP; longer runs stay separate.
- Synonyms (user decision 2026-11-11, replacing the separate REPORT_NLP_SYNONYM table): a REPORT_NLP_WORD row with SOURCE_TEXT (NVARCHAR(128), nullable; WORD_TEXT is NVARCHAR(128)) replaces SOURCE_TEXT with WORD_TEXT/WORD_TAG. WORD_TEXT always means the result word. Scope priority question > project > ALL uses the 100/50/10 scores. The column must exist before deployment. Sources are pre-analyzed (score 200, above every user word) after registering the target as a word with score 0 unless a plain row for the same target already registered it with its scope score, because Kiwi rejects a target morpheme missing from its dictionary such as LG/NNP; a failing row is skipped, never a 503; a spaced source also covers its spaceless form and replaces whole token runs only when no noun of the same word continues the run.
- Stopwords (불용어): REPORT_NLP_WORD rows with WORD_TAG STOP (case-insensitive) are never registered in Kiwi. Any scope excludes the final word (after synonyms and compound merging) for every tag, comparing trimmed, space-collapsed text case-insensitively; SOURCE_TEXT is ignored on STOP rows.
- User dictionary words and synonym targets take precedence over everything: they are never merged with neighbors.
- The original analyze(project, qid, responses) and tokenize(texts, dictionary) overloads keep the previous behavior (synonym rows ignored; no stopwords or merging). Only the statistics API uses the new options.

View File

@@ -246,6 +246,8 @@ When implementation is complete, report in Korean:
- For HTTP/2 header safety and original upstream error propagation, use the shared HTTP/bypass features documented in the routed platform-capabilities reference instead of implementing consumer-local header filters and raw-response exception handlers. - For HTTP/2 header safety and original upstream error propagation, use the shared HTTP/bypass features documented in the routed platform-capabilities reference instead of implementing consumer-local header filters and raw-response exception handlers.
- Response customizers may call `requireSuccess()`; its upstream exception propagates unchanged. Response charset parsing ignores charset-like text inside other quoted header parameters. - Response customizers may call `requireSuccess()`; its upstream exception propagates unchanged. Response charset parsing ignores charset-like text inside other quoted header parameters.
- When Swagger auto-configuration is active, `RequestUserContext` controller arguments are excluded from request input documentation; server-side authentication and client IP injection remain unchanged. - When Swagger auto-configuration is active, `RequestUserContext` controller arguments are excluded from request input documentation; server-side authentication and client IP injection remain unchanged.
- `RequestUserContextProvider` is a public injectable API for resolving the same request identity and client IP used by controller arguments. Its default registration requires Web integration and JwtService; a consumer provider overrides it. Existing principal/details/verified-Bearer fallback, empty-context behavior, resolver overrides and the JwtService resolver constructor remain unchanged. The provider does not enforce authentication or business authorization.
- ExcelMapper's Path-based readStreaming and readHeaderStreaming use PackageAccess.READ, preserving source bytes, size, and modification time, including repeated reads and callback failures. Read-only files are supported, and file handles are released on success or failure; option validation, exception handling, and SAX streaming remain unchanged.
- Do not modify KriCommon or another consumer repository unless it is explicitly included in the approved task scope. - Do not modify KriCommon or another consumer repository unless it is explicitly included in the approved task scope.
### Required Shared References ### Required Shared References

View File

@@ -148,13 +148,19 @@ ProjectMapper는 list/summary/mustViewPkeys를 유지하고 ProgressMapper/Progr
필터는 `filter`의 AST로 입력한다. 그룹 AND/OR/NOT는 `children`, 조건 EQ/NE/GT/GE/LT/LE/IN/NOT_IN/BETWEEN/IS_ANSWERED/IS_UNANSWERED는 `dimensionId`, `reference=QID|VARIABLE`, `values`를 사용한다. JSON 숫자는 수치 비교, 문자열은 기존 SQL collation처럼 대소문자 무시 비교다. 부정 조건은 일치 응답이 하나라도 있으면 제외하고 해당 키가 없는 응답은 포함하는 기존 NOT EXISTS 의미를 유지한다. IS_ANSWERED는 기존 IS NOT NULL 의미여서 빈 문자열도 포함한다. 필터 문자열을 SQL에 삽입하지 않는다. 필터 깊이는 32, 그룹당 자식은 128까지 허용한다. 필터는 `filter`의 AST로 입력한다. 그룹 AND/OR/NOT는 `children`, 조건 EQ/NE/GT/GE/LT/LE/IN/NOT_IN/BETWEEN/IS_ANSWERED/IS_UNANSWERED는 `dimensionId`, `reference=QID|VARIABLE`, `values`를 사용한다. JSON 숫자는 수치 비교, 문자열은 기존 SQL collation처럼 대소문자 무시 비교다. 부정 조건은 일치 응답이 하나라도 있으면 제외하고 해당 키가 없는 응답은 포함하는 기존 NOT EXISTS 의미를 유지한다. IS_ANSWERED는 기존 IS NOT NULL 의미여서 빈 문자열도 포함한다. 필터 문자열을 SQL에 삽입하지 않는다. 필터 깊이는 32, 그룹당 자식은 128까지 허용한다.
주관식 문자열은 `indexes=["QID_입력칸"]`으로 같은 API를 호출한다. `type=TEXT_ANALYSIS`와 `words[{text,tag,count}]`를 반환하며 원문·IDKEY와 기존의 가짜 제목 데이터 행을 노출하지 않는다. 기존 REPORT_NLP_TAG/WORD의 전체·프로젝트·문항 범위 및 사전 점수 10/50/100, SL 포함·품사/길이 조건, 같은 단어의 품사별 별도 빈도와 빈도 내림차순/동률 최초 등장 순서/상위 100개를 유지한다. 문장별 분리는 토큰을 빠뜨리지 않으므로 결과를 전체 토큰 빈도로 집계한다. 별도의 불용어 규칙을 추가하지 않는다. 주관식 문자열은 `indexes=["QID_입력칸"]`으로 같은 API를 호출한다. `type=TEXT_ANALYSIS`와 `words[{text,tag,count}]`를 반환하며 원문·IDKEY와 기존의 가짜 제목 데이터 행을 노출하지 않는다. 기존 REPORT_NLP_TAG/WORD의 전체·프로젝트·문항 범위 및 사전 점수 10/50/100, SL 포함·품사/길이 조건, 빈도 내림차순/동률 최초 등장 순서/상위 100개를 유지한다. 문장별 분리는 토큰을 빠뜨리지 않으므로 결과를 전체 토큰 빈도로 집계한다.
사용자 결정에 따라 최신 공식 KiwiJava·모델 **0.24.0**을 사용한다. 기존 DLL은 실제 실행에서 0.15.2로 확인되었으며 기존 모델은 Java 0.20.4에서도 로딩되지 않았다. 최신 모델 적용에 따른 형태소 결과 변화는 허용하되 Report DB 사전·품사 선택·빈도 집계 규칙은 유지한다. 0.24.0의 CONG 모델, default/typo dictionary 로딩, 이형태 통합 비활성, 오타 교정 비활성, match=0을 사용한다. 모델은 `kri.kart.nlp.model-directory`(기본 `./nlp_model/v0.24.0`)에 별도로 설치한다. `scripts/setup-kiwi-model.ps1`은 공식 모델 다운로드·SHA-256 검증 후 비어 있는 폴더에만 설치한다. 모델은 Git에서 제외한다. Kiwi 라이브러리/모델이 없거나 분석에 실패하면 해당 문항에 status=503, code=KIWI_NLP_UNAVAILABLE인 error 이벤트를 반환하고 다음 문항을 계속한다. 텍스트 분석 요청 옵션: `wordMergeMode`는 같은 글자의 품사별 빈도 합산 방식으로 1=모두 합치기, 2=명사 계열(NNG·NNP·NNB·NR·NP·SL·SH)만 합치기(기본값), 3=합치지 않기이다. 합친 단어는 가장 많이 나온 품사(동률이면 먼저 나온 품사)를 사용한다. `compoundNounMaxTokens`(1~5, 기본값 3, 1이면 끔)는 명사 접두사·접미사를 붙이고, 한 어절 안에서 공백 없이 붙은 NNG·NNP·SL 토큰(한글 명사 1개 이상 포함)을 복합명사 하나로 합친다. 하나라도 NNP면 NNP이고, 상한을 넘는 묶음은 합치지 않는다. 사용자 사전 단어와 치환 결과는 합치지 않는다.
불용어: REPORT_NLP_WORD에서 WORD_TAG가 `STOP`인 행은 불용어이며 Kiwi 사전에 등록하지 않는다. 치환·복합명사 합치기 후의 최종 단어를 품사와 관계없이 제외하며, 앞뒤·연속 공백과 대소문자를 구분하지 않는다. 전체·프로젝트·문항 중 어느 범위에 있어도 적용한다. 그 밖의 불용어 규칙은 두지 않는다.
치환: REPORT_NLP_WORD에서 `SOURCE_TEXT`가 있는 행은 `SOURCE_TEXT`를 `WORD_TEXT`/`WORD_TAG`로 바꿔 집계한다(예: SOURCE_TEXT 엘지 → WORD_TEXT LG). `WORD_TEXT`는 모든 행에서 결과에 나오는 단어이며, 치환 행의 `WORD_TEXT`는 점수 0으로만 등록한다(같은 단어의 일반 행이 있으면 그 범위 점수 유지). 우선순위는 문항 > 프로젝트 > 전체이고 사용자 사전보다 우선한다. 띄어 쓴 원표기는 붙여 쓴 형태도 함께 처리하며, 같은 어절의 명사가 이어지면 치환하지 않는다. 짧은 원표기는 긴 단어 안에서도 바뀌므로(엘지전자 → LG + 전자) 필요하면 긴 표기 행을 함께 둔다. `SOURCE_TEXT` 컬럼(NVARCHAR(128), NULL 허용)이 없는 DB에서는 사전 조회가 실패하므로 컬럼 추가 후 배포한다.
사용자 결정에 따라 최신 공식 KiwiJava·모델 **0.24.0**을 사용한다. 기존 DLL은 실제 실행에서 0.15.2로 확인되었으며 기존 모델은 Java 0.20.4에서도 로딩되지 않았다. 최신 모델 적용에 따른 형태소 결과 변화는 허용하되 Report DB 사전·품사 선택·빈도 집계 규칙은 유지한다. 0.24.0의 CONG 모델과 기본 빌드 옵션(default·typo·multi-word dictionary, 이형태 통합), `Match.allWithNormalizing | oovChrModel`, 오타 교정(basicTypoSetWithContinual, 기준값 2.5)을 사용한다. 같은 사용자 사전 단어는 범위 중 가장 높은 점수로 한 번만 등록한다. 모델은 `kri.kart.nlp.model-directory`(기본 `./nlp_model/v0.24.0`)에 별도로 설치한다. `scripts/setup-kiwi-model.ps1`은 공식 모델 다운로드·SHA-256 검증 후 비어 있는 폴더에만 설치한다. 모델은 Git에서 제외한다. Kiwi 라이브러리/모델이 없거나 분석에 실패하면 해당 문항에 status=503, code=KIWI_NLP_UNAVAILABLE인 error 이벤트를 반환하고 다음 문항을 계속한다.
Gradle은 Windows에서 win-x64, Linux에서 lnx-x86-64 또는 lnx-aarch64 JAR을 선택한다. Windows에서 Linux 배포물을 만들 때 `./gradlew.bat bootJar -PkiwiPlatform=lnx-x86-64`로 명시한다. 배포 JAR과 JVM 플랫폼이 일치해야 하고 모델도 운영에 별도로 배치한다. Kiwi 및 모델의 라이선스는 공식 저장소의 LGPL-2.1 조건을 확인한다: https://github.com/bab2min/Kiwi/tree/v0.24.0/bindings/java . Gradle은 Windows에서 win-x64, Linux에서 lnx-x86-64 또는 lnx-aarch64 JAR을 선택한다. Windows에서 Linux 배포물을 만들 때 `./gradlew.bat bootJar -PkiwiPlatform=lnx-x86-64`로 명시한다. 배포 JAR과 JVM 플랫폼이 일치해야 하고 모델도 운영에 별도로 배치한다. Kiwi 및 모델의 라이선스는 공식 저장소의 LGPL-2.1 조건을 확인한다: https://github.com/bab2min/Kiwi/tree/v0.24.0/bindings/java .
검증: 원본 C# Report.cs를 실행한 합성 비교 기준 25개, 필터/좌표/조회 분리/인증·Swagger/API와 실제 Windows Kiwi 0.24.0 분석을 테스트한다. 네이티브 검증은 `./gradlew.bat test '-PkiwiModelDirectory=nlp_model/v0.24.0'`로 실행한다. 이 속성이 없으면 해당 네이티브 테스트 1개만 생략한다. `build/phase5-openapi.json`은 API 테스트에서 생성된다. 실제 SQL Server 결과, Ubuntu 네이티브 실행·운영 성능은 별도 검수 대상이며 합성 테스트 통과로 운영 동등성을 주장하지 않는다. 검증: 원본 C# Report.cs를 실행한 합성 비교 기준 25개, 필터/좌표/조회 분리/인증·Swagger/API와 실제 Windows Kiwi 0.24.0 분석을 테스트한다. 네이티브 검증은 `./gradlew.bat test '-PkiwiModelDirectory=nlp_model/v0.24.0'`로 실행한다. 이 속성이 없으면 실제 모델을 사용하는 네이티브 테스트만 생략한다. `build/phase5-openapi.json`은 API 테스트에서 생성된다. 실제 SQL Server 결과, Ubuntu 네이티브 실행·운영 성능은 별도 검수 대상이며 합성 테스트 통과로 운영 동등성을 주장하지 않는다.
### 배열 집계와 NDJSON 스트리밍 ### 배열 집계와 NDJSON 스트리밍

View File

@@ -55,7 +55,7 @@ configurations.configureEach {
} }
dependencies { dependencies {
implementation 'com.kri.common:starter:beta.1.7.0' implementation 'com.kri.common:starter:beta.1.7.1'
implementation 'org.apache.poi:poi-ooxml:5.5.1' implementation 'org.apache.poi:poi-ooxml:5.5.1'
implementation "kr.pe.bab2min:kiwi-java:${kiwiVersion}:${kiwiPlatform}@jar" implementation "kr.pe.bab2min:kiwi-java:${kiwiVersion}:${kiwiPlatform}@jar"

245
docs/statistics-flow.md Normal file
View File

@@ -0,0 +1,245 @@
# 통계 데이터와 계산 흐름
현재 구현은 **문항과 응답을 요청당 한 번 준비하고, 요청한 문항별로 응답을 선택해 계산한 뒤 결과를 순서대로 전송**한다.
- 기준: 2026-10-23 소스 코드. 실제 DB 결과를 비교한 문서는 아니다.
- 대상 API: `POST /api/v1/project/{projectType}/{pkey}/statistics/result`
- `index`: 집계할 문항. 예: 만족도.
- `banner`: 응답을 나누는 기준. 예: 성별, 연령대.
- 그림의 `클래스.메서드()`는 실제 구현 이름이다. 같은 클래스의 연속 작업은 한 상자에 묶었다.
- 1번은 간략 버전, 2번은 상세 버전이다. 상세 버전은 큰 그림 하나가 길어지지 않도록 단계별로 나눴다.
## 1. 간략 버전
```mermaid
flowchart TD
REQ["화면에서 집계 요청<br/>StatisticsController.result()"]
QUESTIONS[("KON 또는 KONKIWI<br/>문항 정의")]
LOAD["문항과 응답을 한 번 준비<br/>StatisticsService.prepare()"]
RESPONSES[("KON 또는 KONKIWI<br/>원래 응답 + 저장된 파생 응답<br/>필요한 경우 KIWI 프로필 응답")]
FILTER["응답 JSON을 Map으로 변환하고 필터 적용<br/>StatisticsDataService.normalize()<br/>StatisticsFilterService.compile()로 만든 조건 사용"]
CALC["요청한 문항별로 대상 응답 선택<br/>StatisticsService.calculate()"]
TYPE{"주관식 문자 문항인가?<br/>StatisticsService.isOpenText()"}
TABLE["전체·배너별 N / % / 통계량 계산<br/>StatisticsAggregationService.aggregate()"]
LAYOUT["표 머리글·병합·전치 적용<br/>StatisticsTableService.table()"]
NLP["형태소 분석과 단어 빈도 계산<br/>StatisticsTextService.analyze()"]
DICT[("Report DB: NLP 사전·품사 설정<br/>로컬 파일: Kiwi 모델")]
OUT["문항별 result 또는 error 전송<br/>StatisticsService.stream()<br/>StatisticsStreamService.send()"]
END["마지막 complete 전송<br/>전체 수·성공 수·실패 수"]
REQ --> LOAD
QUESTIONS --> LOAD
RESPONSES --> LOAD
LOAD --> FILTER --> CALC --> TYPE
TYPE -->|아니오| TABLE --> LAYOUT --> OUT
TYPE -->|예| NLP --> OUT
DICT --> NLP
OUT -->|다음 문항| CALC
OUT -->|모든 문항 처리 후| END
```
예를 들어 **“30대 응답자 중 성별에 따른 만족도”**를 요청하면, 나이로 응답자를 걸러 낸 다음 만족도 응답을 성별로 나눠 N·%·평균 등을 계산한다.
## 2. 상세 버전
### 2-1. 공통 준비: 어디서 데이터를 가져오는가
이 단계는 스트림을 반환하기 전에 실행한다. 문항 목록과 합집합으로 선택한 응답 데이터는 요청당 한 번 조회한다. 다만 텍스트 분석용 사전·품사 설정은 이후 텍스트 문항마다 별도로 조회한다.
```mermaid
flowchart TD
API["StatisticsController.result()<br/>StatisticsBatchRequestDto.requests()<br/>indexes를 개별 요청 목록으로 변환"]
PREP["StatisticsService.prepare()<br/>validate(): 모드·index·날짜 확인"]
QLOAD["StatisticsService.loadQuestions()<br/>QuestionRepository.list()<br/>QuestionMapper.list()"]
SESSION["Repository가 SurveySessionProvider.mapper() 호출<br/>forProject(): 0=KON / 1=KONKIWI"]
QDB[("선택한 설문 DB<br/>QUESTION + DERIVED_VARIABLE<br/>KIWI이며 role=1001이면 프로필 문항 포함")]
ORGANIZE["QuestionService.organize()<br/>expandOpen() / expandRank() / expandAttributes()<br/>입력칸·순위·속성별 문항으로 확장"]
VALID["StatisticsService.validateDimensions()<br/>StatisticsDimensionService.resolve()<br/>모든 index와 banner의 문항 정의 확인"]
COMPILE["StatisticsFilterService.compile()<br/>필터 구조를 응답자 판정 함수로 변환"]
ROOTS["StatisticsService.loadResponses()<br/>storageRoots() / toStorageRoot()<br/>모든 요청의 index·banner 루트 합집합 생성"]
RAW[("선택한 설문 DB<br/>DATA.DATJSON: 원래 응답<br/>DERIVED_DATA.DATJSON: 저장된 파생 응답")]
PROFILE[("KIWI_SURVEY.dbo.V_PROFILE<br/>P_DATJSON: 프로필 응답")]
QUERY["StatisticsRepository.data()<br/>StatisticsMapper.data()<br/>완료·조사 유형·날짜 조건으로 조회하고 JSON 결합"]
DAT["StatisticsDataDto.dat<br/>집계용: 선택한 루트가 들어 있는 저장 그룹"]
FDAT["StatisticsDataDto.filterDat<br/>필터용: 전체 응답 그룹<br/>필터가 없으면 null"]
NORMAL["StatisticsDataService.normalize() → flatten()<br/>JSON 키를 대문자로 통일하고 Map으로 변환<br/>SurveyResponseDto 생성"]
APPLY["StatisticsService.loadResponses()의 stream().filter(filter)<br/>판정 함수는 response.filterValues() 사용<br/>조건을 통과한 응답만 남김"]
BATCH["StatisticsBatchDto 생성<br/>project / requests / questions / responses"]
AUDIT["StatisticsService.prepare() → LogService.writeSiteLog()<br/>Get Report Data를 배치당 한 번 기록 시도<br/>그 후 Controller가 스트림 반환"]
API --> PREP --> QLOAD
SESSION -. "문항 조회와 응답 조회에 각각 적용" .-> QLOAD
SESSION -.-> QUERY
QDB --> QLOAD --> ORGANIZE --> VALID --> COMPILE --> ROOTS
ROOTS --> QUERY
RAW --> QUERY
PROFILE -. "needsProfileResponses()가 참조 여부 확인" .-> QUERY
QUERY --> DAT --> NORMAL
QUERY --> FDAT --> NORMAL
COMPILE -. "판정 함수" .-> APPLY
NORMAL --> APPLY --> BATCH --> AUDIT
ORGANIZE -. "questions" .-> BATCH
```
**SQL에서 거르는 조건** — [StatisticsMapper.xml](../src/main/resources/mybatis/mapper/StatisticsMapper.xml)의 `data`:
| 조건 | 실제 처리 |
|---|---|
| 프로젝트 | `DATA.PKEY = pkey` |
| 완료 상태 | 모든 dataScope에서 `Status = 10` |
| `ACTUAL` | `SURVEY_TYPE <> 'TEST'` |
| `TEST` | `SURVEY_TYPE = 'TEST'` |
| `ALL` | 조사 유형 제한 없음. 완료 조건은 유지 |
| 날짜 | `FN_LAST_TIME(survey_date)`를 날짜로 변환해 시작일·종료일 포함 비교. 시작일만 있으면 그 하루 |
| 파생 응답 | `DERIVED_DATA`를 `PKEY + IDKEY`로 LEFT JOIN. 이 요청에서 파생값을 재계산하지 않음 |
| 프로필 응답 | KIWI이며 index·banner·filter에서 PF/PS 문항을 참조하면 `V_PROFILE` JOIN |
**집계용 데이터와 필터용 데이터를 분리하는 이유:** 만족도만 집계해도 나이로 필터링할 수 있어야 하기 때문이다.
| 데이터 | 사용하는 곳 | 의미 |
|---|---|---|
| `dat` → `SurveyResponseDto.values()` | 문항별 응답 선택, N·%·통계량, 텍스트 분석 | 선택한 저장 그룹의 응답. 요청한 키만 잘라 내는 방식은 아니므로 같은 그룹의 다른 키도 포함될 수 있음 |
| `filterDat` → `SurveyResponseDto.filterValues()` | `StatisticsFilterService.compileCondition()`이 만든 판정 함수 | 전체 응답을 보고 필터 판단. 필터가 없으면 normalize 단계에서 `values`를 함께 사용 |
루트는 저장된 응답 그룹을 찾는 기준이다. `toStorageRoot()`는 대문자로 바꾸고 `#A`를 제거한 뒤 첫 `_` 앞부분을 사용한다. 예를 들어 `q1_2#a`는 `Q1`이 된다. TABULATION 모드에서는 index 대신 `INDEX` 루트를 사용한다.
### 2-2. 문항별 계산: 어떤 응답으로 N·%를 만드는가
준비한 배치를 `StatisticsStreamService.stream()`이 비동기 작업에 넘긴다. `writeEvents()` → `StatisticsService.stream()` 순서로 들어가며, 아래 계산을 indexes의 입력 순서대로 실행한다. 같은 index가 반복되어도 해당 위치마다 처리한다.
```mermaid
flowchart TD
BATCH["공통 준비 결과<br/>StatisticsBatchDto"]
LOOP["StatisticsService.stream()<br/>이번 위치의 요청 선택 → calculate()"]
PICK["StatisticsService.withinStorageRoots()<br/>이번 index·banner의 루트에 해당하는 키가<br/>하나라도 있는 응답자만 선택"]
DIM["StatisticsDimensionService.resolve()<br/>문항 ID → 기본 키·보기·속성·순위 범위<br/>숫자 문항 보기는 실제 응답에서 구성"]
TYPE{"StatisticsService.isOpenText()<br/>text 문항이며 #A가 아닌가?"}
TEXT["2-3의 텍스트 분석으로 이동<br/>StatisticsService.textAnalysis()"]
AGG["StatisticsAggregationService.aggregate()<br/>columns()로 열별 응답 목록 생성"]
TOTAL["StatisticsSelectionService.total() / cases()<br/>전체 응답 목록과 보기별 응답 목록"]
BANNER["StatisticsAggregationService.addBanner()<br/>StatisticsSelectionService.bannerTotal() / bannerCases()<br/>배너 값별로 목록 분리, 중첩 배너는 재귀 처리"]
MODE{"집계 모드"}
NORMAL["StatisticsAggregationService<br/>totalRow() / addChoiceRows()<br/>countCell() / percentBase()<br/>N=목록 크기, %=N÷분모×100"]
SUMMARY["StatisticsAggregationService.addSummaryRows()<br/>숫자: 평균·중앙값·표준편차 등<br/>척도: Top/Mid/Bottom·NPS 또는 평균<br/>요청한 사용자 값 그룹 추가"]
CROSS["StatisticsAggregationService.bannerCrossTable()<br/>crossCount() / overlapCount(): 배너 간 겹침<br/>crossBase() / groupBase(): 분모 결정"]
TABLE["StatisticsTableService.table()<br/>headers(): 머리글과 병합 정보 생성<br/>필요하면 transpose(): 행·열 교환<br/>StatisticsResponseDto, type=TABLE"]
EVENT["StatisticsService.resultEvent() / errorEvent()<br/>문항 계산 오류는 해당 위치의 error로 변환"]
SEND["StatisticsStreamService.send()<br/>JSON 한 줄 + 줄바꿈 전송"]
DONE["StatisticsService.completeEvent()<br/>total / succeeded / failed"]
BATCH --> LOOP --> PICK --> DIM --> TYPE
TYPE -->|예| TEXT --> EVENT
TYPE -->|아니오| AGG --> TOTAL --> BANNER --> MODE
MODE -->|STANDARD 또는 TABULATION_0| NORMAL --> SUMMARY --> TABLE
MODE -->|TABULATION_1| CROSS --> TABLE
TABLE --> EVENT --> SEND
SEND -->|다음 위치| LOOP
SEND -->|모두 처리함| DONE
```
`withinStorageRoots()`는 **응답자를 고르는 단계**다. 응답자의 Map에서 다른 키를 삭제하는 단계는 아니다. 전체 배치를 한 번에 조회한 영향으로 특정 문항의 대상자가 늘어나지 않도록 다시 제한한다.
계산 도중 연결이 끊기거나 취소되면 반복과 전송을 중단한다. 공통 준비 단계의 오류는 스트림 시작 전 HTTP 오류이며, 문항별 계산 오류와 구분된다.
**N과 분모가 만들어지는 위치:**
| 항목 | 실제 클래스·메서드 | 계산 의미 |
|---|---|---|
| 전체 사례수 | `StatisticsSelectionService.total()` | 해당 문항에 응답한 목록 구성 |
| 보기별 N | `StatisticsSelectionService.cases()` | 해당 보기를 선택한 목록의 크기 |
| 배너별 사례수와 N | `bannerTotal()` / `bannerCases()` | 상위 단계 목록을 배너의 보기 조건으로 다시 선택 |
| 일반 비율 | `StatisticsAggregationService.countCell()` | 분모가 0이면 0%, 아니면 N ÷ 분모 × 100 |
| 일반 표의 Total 비율 | `totalRow()` | 해당 열의 보기별 비율 합. 복수응답에서는 100%를 넘을 수 있음 |
| 평균 | `mean()` | 보기 값 × 사례수의 합 ÷ 결측 사례를 뺀 분모. 역척도 처리 포함 |
| 나머지 숫자 통계 | `numericValues()` → `calculate()` | 응답 숫자 목록으로 Median·SD·Max·Min·Mode·선택적 Sum 계산 |
| 척도 그룹과 NPS | `addScaleGroups()` / `addGroup()` / `addNpsRow()` | 그룹에 해당하는 보기 사례 합산. NPS는 Promoters % − Detractors % |
| 반올림·표시 문자열 | `StatisticsArithmetic.round()` / `display()` | 계산값과 화면 표시 문자열 구성 |
순위·속성 응답 선택은 `StatisticsSelectionService.select()`가 여러 슬롯을 순회해 목록에 추가한다. **같은 응답자가 여러 슬롯에 들어가면 여러 번 집계될 수 있으므로 N이 항상 중복 없는 사람 수인 것은 아니다.** 속성 전체 문항의 요약은 `columns()`에서 요청 배너 대신 해당 문항의 `#A` 배너를 사용한다.
**모드별 차이:**
| 모드 | 표 구성 | 분모 결정 메서드와 규칙 |
|---|---|---|
| `STANDARD` | 집계 문항의 보기를 행으로, 배너를 열로 구성 | `percentBase()`: 해당 열의 전체 응답 목록 크기 |
| `TABULATION_0` | 빈 index로 전체 표본을 놓고 배너 구성비 계산 | `percentBase()`: Total 열은 자체 사례수, 배너 열은 형제 열 사례수의 합. 형제가 없으면 Total 사례수. 형제 판정은 실제 열 ID의 마지막 한 자리 숫자 규칙을 유지 |
| `TABULATION_1` | 배너를 행과 열 양쪽에 놓고 교차 계산 | `crossBase()`: Total 열은 행의 형제 열들을 합친 고유 IDKEY 수, 나머지 열은 해당 열의 사례수 |
`TABULATION_1`의 `overlapCount()`는 같은 IDKEY의 좌우 목록 쌍을 센다. 모든 교차 셀이 고유 응답자 교집합 크기와 같다고 가정하면 안 된다. 빈 분모의 일반 빈도 셀은 0%이며, 정의되지 않는 그룹 비율·통계량은 null과 `-` 표시를 사용한다.
### 2-3. 텍스트 분석: 어떤 사전으로 단어를 세는가
```mermaid
flowchart TD
INPUT["StatisticsService.textAnalysis()<br/>StatisticsTextService.analyze()"]
TEXT["StatisticsTextService.answerTexts()<br/>해당 QID와 QID_ 하위 키의 문자열 수집<br/>HTML 문자 복원"]
DICT[("Report DB<br/>REPORT_NLP_WORD (STOP 행 = 불용어, SOURCE_TEXT 행 = 치환)<br/>REPORT_NLP_TAG")]
REPO["NlpRepository.words() / tags()<br/>NlpMapper.words() / tags()<br/>kartSqlSessionTemplate 사용"]
MODEL[("로컬 Kiwi 모델 파일<br/>kri.kart.nlp.model-directory")]
TOKEN["KiwiNlpService.tokenize()<br/>응답 문자열 + 사용자 사전 + 치환 → 형태소 목록<br/>오타 교정·구어체 정규화<br/>compoundNounMaxTokens에 따라 복합명사 합치기"]
FILTER["StatisticsTextService.analyze() / isCounted()<br/>STOP 불용어 제외<br/>선택 품사 중 명사 또는 2글자 이상<br/>SL은 선택 품사·길이와 관계없이 포함"]
COUNT["StatisticsTextService.analyze() / merge()<br/>단어+품사별 빈도 합산<br/>wordMergeMode에 따라 같은 글자 합산<br/>빈도 내림차순, 동률은 첫 등장 순서<br/>상위 100개 선택"]
OUT["StatisticsResponseDto, type=TEXT_ANALYSIS<br/>words: 단어·품사·빈도<br/>문항별 result 이벤트로 반환"]
INPUT --> TEXT --> TOKEN
DICT --> REPO
REPO -->|사전 단어·치환| TOKEN
MODEL --> TOKEN
TOKEN --> FILTER --> COUNT --> OUT
REPO -->|선택 품사·불용어| FILTER
```
사전은 전체 공통(`PKEY=ALL`), 프로젝트 공통(`QID=ALL`), 해당 문항 범위를 합쳐 읽는다. 사용자 단어의 점수는 각각 10·50·100이며, 같은 단어는 가장 높은 점수로 한 번만 등록한다. 이 점수는 형태소 분석 사전에 사용하는 값이며 최종 빈도에 곱하는 통계 가중치가 아니다. 모델이나 native 실행 환경이 없으면 오류를 반환한다.
텍스트 분석 처리 순서는 다음과 같다.
1. **치환**: `REPORT_NLP_WORD`에서 `SOURCE_TEXT`가 있는 행은 `SOURCE_TEXT`를 `WORD_TEXT`/`WORD_TAG`로 바꾼다. 우선순위는 문항 > 프로젝트 > 전체이고, 사용자 사전보다 우선한다.
2. **복합명사 합치기**: `compoundNounMaxTokens`(기본 3)에 따라 한 어절 안에서 붙은 명사를 합친다. 사용자 사전 단어와 치환 결과는 합치지 않는다.
3. **불용어 제외**: `REPORT_NLP_WORD`에서 WORD_TAG가 `STOP`인 단어를 품사와 관계없이 제외한다. 대소문자와 공백 차이는 무시한다.
4. **품사 합산**: `wordMergeMode`(기본 2=명사 계열)에 따라 같은 글자의 품사별 빈도를 합친다.
기존 `analyze(project, qid, responses)`와 `tokenize(texts, dictionary)` 오버로드는 치환·불용어·합치기를 적용하지 않는 이전 동작을 유지하며, 통계 API만 새 옵션을 사용한다.
### 2-4. 쿼터는 어디서 갈라지는가
일반 통계 스트림과 별도인 `POST /api/v1/quota/{projectType}/{pkey}/{quotaOrder}/result`는 저장된 설정으로 집계한 뒤 목표 수와 결합한다. 단일 JSON 응답이다.
| 순서 | 실제 클래스·메서드 | 데이터 처리 |
|---|---|---|
| 1 | `QuotaService.result()` → `QuotaRepository.quotaSet()` → `QuotaMapper.quotaSet()` | 선택한 설문 DB의 `QUOTASET`에서 설정 조회 |
| 2 | `QuotaService.result()` → `LogService.writeSiteLog()` | 설정을 찾은 뒤 `QUOTA VIEW` 기록 시도 |
| 3 | `QuotaService.result()` / `LegacyFilterService.parse()` | 첫 ROW를 index로, COL을 배너로 구성. 저장된 FILTER는 AST로 해석. SQL로 실행하지 않음 |
| 4 | `StatisticsService.prepareQuota()` | 2-1의 공통 준비 재사용. 빈 STANDARD index 허용. `Get Report Data`는 기록하지 않음 |
| 5 | `StatisticsAggregationService.aggregate()` | 사례수 표 계산. 소계 없이 집계하며 일반 통계의 `stream()` / `calculate()`를 경유하지 않음 |
| 6 | `QuotaRepository.quotaEntries()` → `QuotaMapper.quotaEntries()` | 같은 설문 DB의 `QUOTA`에서 목표 수와 설정 조회 |
| 7 | `QuotaComputationService.compute()` | 계산된 사례수와 `ORI_CNT` 목표 수 결합. 0 목표 행 처리·Total 재계산·달성률 및 표시 상태 구성 |
## 3. 소스 바로가기
링크는 이 문서 기준 상대 경로다. private 메서드도 흐름 설명에 포함했다.
| 소스 | 먼저 볼 메서드 |
|---|---|
| [StatisticsController.java](../src/main/java/com/kri/kart/server/controller/StatisticsController.java) | `result()` |
| [StatisticsBatchRequestDto.java](../src/main/java/com/kri/kart/server/dto/statistics/StatisticsBatchRequestDto.java) | `requests()` |
| [StatisticsService.java](../src/main/java/com/kri/kart/server/service/StatisticsService.java) | `prepare()` → `loadQuestions()` / `loadResponses()`, `stream()` → `calculate()` |
| [SurveySessionProvider.java](../src/main/java/com/kri/kart/server/persistence/SurveySessionProvider.java) | `mapper()` / `forProject()` |
| [QuestionService.java](../src/main/java/com/kri/kart/server/service/QuestionService.java) | `organize()` / `expandOpen()` / `expandRank()` / `expandAttributes()` |
| [QuestionRepository.java](../src/main/java/com/kri/kart/server/persistence/repository/QuestionRepository.java) / [QuestionMapper.xml](../src/main/resources/mybatis/mapper/QuestionMapper.xml) | `list()` / statement `list` |
| [StatisticsRepository.java](../src/main/java/com/kri/kart/server/persistence/repository/StatisticsRepository.java) / [StatisticsMapper.xml](../src/main/resources/mybatis/mapper/StatisticsMapper.xml) | `data()` / statement `data` |
| [StatisticsDataService.java](../src/main/java/com/kri/kart/server/service/statistics/StatisticsDataService.java) | `normalize()` / `flatten()` |
| [StatisticsFilterService.java](../src/main/java/com/kri/kart/server/service/statistics/StatisticsFilterService.java) | `compile()` / `compileGroup()` / `compileCondition()` |
| [StatisticsDimensionService.java](../src/main/java/com/kri/kart/server/service/statistics/StatisticsDimensionService.java) | `resolve()` / `choices()` / `answeredChoices()` |
| [StatisticsSelectionService.java](../src/main/java/com/kri/kart/server/service/statistics/StatisticsSelectionService.java) | `total()` / `cases()` / `bannerTotal()` / `bannerCases()` / `select()` |
| [StatisticsAggregationService.java](../src/main/java/com/kri/kart/server/service/statistics/StatisticsAggregationService.java) | `aggregate()` / `columns()` / `addBanner()` / `percentBase()` / `bannerCrossTable()` |
| [StatisticsArithmetic.java](../src/main/java/com/kri/kart/server/service/statistics/StatisticsArithmetic.java) | `round()` / `display()` |
| [StatisticsTableService.java](../src/main/java/com/kri/kart/server/service/statistics/StatisticsTableService.java) | `table()` / `headers()` / `transpose()` |
| [StatisticsTextService.java](../src/main/java/com/kri/kart/server/service/statistics/StatisticsTextService.java) | `analyze()` / `answerTexts()` / `isCounted()` |
| [KiwiNlpService.java](../src/main/java/com/kri/kart/server/service/statistics/KiwiNlpService.java) | `tokenize()` |
| [NlpRepository.java](../src/main/java/com/kri/kart/server/persistence/repository/NlpRepository.java) / [NlpMapper.xml](../src/main/resources/mybatis/mapper/NlpMapper.xml) | `words()` / `tags()` |
| [StatisticsStreamService.java](../src/main/java/com/kri/kart/server/service/statistics/StatisticsStreamService.java) | `stream()` / `writeEvents()` / `send()` |
| [QuotaService.java](../src/main/java/com/kri/kart/server/service/QuotaService.java) | `result()` |
| [QuotaRepository.java](../src/main/java/com/kri/kart/server/persistence/repository/QuotaRepository.java) / [QuotaMapper.xml](../src/main/resources/mybatis/mapper/QuotaMapper.xml) | `quotaSet()` / `quotaEntries()` |
| [QuotaComputationService.java](../src/main/java/com/kri/kart/server/service/quota/QuotaComputationService.java) | `compute()` |
Mermaid 지원 Markdown 미리보기에서 다이어그램을 볼 수 있다. 문서의 화살표는 주요 처리·데이터 전달 관계를 나타내며 모든 화살표가 직접적인 Java 메서드 호출을 의미하지는 않는다.

View File

@@ -3,3 +3,6 @@
org.gradle.jvmargs=-Xmx2g org.gradle.jvmargs=-Xmx2g
org.gradle.parallel=true org.gradle.parallel=true
org.gradle.caching=true org.gradle.caching=true
# Enabled parallel sync for Gradle 9.4+
org.gradle.tooling.parallel=true

View File

@@ -100,6 +100,8 @@ public final class KartConstant {
// StatisticsTextService // StatisticsTextService
public static final int TOP_WORD_LIMIT = 100; public static final int TOP_WORD_LIMIT = 100;
public static final String FOREIGN_WORD_TAG = "SL"; public static final String FOREIGN_WORD_TAG = "SL";
// 불용어: REPORT_NLP_WORD rows with this WORD_TAG are excluded from word counts.
public static final String STOPWORD_TAG = "STOP";
// StatisticsController // StatisticsController
public static final String CROSSTAB_PAGE = "/Report/Crosstab"; public static final String CROSSTAB_PAGE = "/Report/Crosstab";

View File

@@ -0,0 +1,32 @@
package com.kri.kart.server.common.enums;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonValue;
import com.kri.common.feature.enums.CodeEnum;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
@Getter(onMethod_ = @JsonValue)
@RequiredArgsConstructor
@Schema(description = "같은 단어의 품사별 빈도 합산: 1=모두 합치기, 2=명사 계열만 합치기, 3=합치지 않기")
public enum WordMergeMode implements CodeEnum<Integer> {
ALL(1),
NOUN(2),
NONE(3);
private final Integer code;
@JsonCreator(mode = JsonCreator.Mode.DELEGATING)
public static WordMergeMode fromCode(Integer code) {
if (code == null) {
return null;
}
for (WordMergeMode s : values()) {
if (s.code.equals(code)) {
return s;
}
}
throw new IllegalArgumentException("Unknown WordMergeMode code: " + code);
}
}

View File

@@ -7,6 +7,7 @@ import org.apache.ibatis.session.SqlSessionFactory;
import org.mybatis.spring.SqlSessionFactoryBean; import org.mybatis.spring.SqlSessionFactoryBean;
import org.mybatis.spring.SqlSessionTemplate; import org.mybatis.spring.SqlSessionTemplate;
import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties; import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Bean;
@@ -19,6 +20,13 @@ import org.springframework.jdbc.datasource.DataSourceTransactionManager;
@EnableConfigurationProperties(KartDatabaseProperties.class) @EnableConfigurationProperties(KartDatabaseProperties.class)
@ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true") @ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true")
public class KartMyBatisConfig { public class KartMyBatisConfig {
private final long leakDetectionThreshold;
public KartMyBatisConfig(
@Value("${spring.datasource.hikari.leak-detection-threshold:0}") long leakDetectionThreshold) {
this.leakDetectionThreshold = leakDetectionThreshold;
}
@Bean(destroyMethod = "close") @Bean(destroyMethod = "close")
public HikariDataSource konDataSource(KartDatabaseProperties properties, Validator validator) { public HikariDataSource konDataSource(KartDatabaseProperties properties, Validator validator) {
return dataSource("kon", properties.getKon(), validator); return dataSource("kon", properties.getKon(), validator);
@@ -53,6 +61,7 @@ public class KartMyBatisConfig {
dataSource.setMaximumPoolSize(source.getMaximumPoolSize()); dataSource.setMaximumPoolSize(source.getMaximumPoolSize());
dataSource.setMinimumIdle(0); dataSource.setMinimumIdle(0);
dataSource.setConnectionTimeout(10000); dataSource.setConnectionTimeout(10000);
dataSource.setLeakDetectionThreshold(leakDetectionThreshold);
dataSource.setInitializationFailTimeout(-1); dataSource.setInitializationFailTimeout(-1);
return dataSource; return dataSource;
} }

View File

@@ -0,0 +1,25 @@
package com.kri.kart.server.config;
import com.kri.common.feature.web.RequestUserContextProvider;
import com.kri.kart.server.interceptor.ProjectAccessInterceptor;
import com.kri.kart.server.service.ProjectService;
import lombok.RequiredArgsConstructor;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
@Configuration(proxyBeanMethods = false)
@RequiredArgsConstructor
@ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true")
public class ProjectAccessConfig implements WebMvcConfigurer {
private final RequestUserContextProvider requestUserContextProvider;
private final ProjectService projectService;
@Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(new ProjectAccessInterceptor(requestUserContextProvider, projectService))
.addPathPatterns("/api/v1/**")
.order(1);
}
}

View File

@@ -1,7 +1,8 @@
package com.kri.kart.server.config; package com.kri.kart.server.config;
import com.kri.common.feature.web.RequestUserContextProvider;
import com.kri.kart.server.interceptor.ExternalAccessInterceptor;
import com.kri.kart.server.service.ShareLinkService; import com.kri.kart.server.service.ShareLinkService;
import com.kri.kart.server.service.share.ExternalAccessInterceptor;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties; import org.springframework.boot.context.properties.EnableConfigurationProperties;
@@ -14,10 +15,13 @@ import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
@RequiredArgsConstructor @RequiredArgsConstructor
@ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true") @ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true")
public class ShareLinkConfig implements WebMvcConfigurer { public class ShareLinkConfig implements WebMvcConfigurer {
private final RequestUserContextProvider requestUserContextProvider;
private final ShareLinkService shareLinkService; private final ShareLinkService shareLinkService;
@Override @Override
public void addInterceptors(InterceptorRegistry registry) { public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(new ExternalAccessInterceptor(shareLinkService)).addPathPatterns("/**"); registry.addInterceptor(new ExternalAccessInterceptor(requestUserContextProvider, shareLinkService))
.addPathPatterns("/**")
.order(0);
} }
} }

View File

@@ -1,5 +1,6 @@
package com.kri.kart.server.controller; package com.kri.kart.server.controller;
import com.kri.common.feature.auth.dto.JwtUserClaimsDto;
import com.kri.common.feature.web.RequestUserContext; import com.kri.common.feature.web.RequestUserContext;
import com.kri.kart.server.dto.auth.LoginRequestDto; import com.kri.kart.server.dto.auth.LoginRequestDto;
import com.kri.kart.server.dto.auth.LoginResponseDto; import com.kri.kart.server.dto.auth.LoginResponseDto;
@@ -7,6 +8,10 @@ import com.kri.kart.server.service.AuthService;
import jakarta.validation.Valid; import jakarta.validation.Valid;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.http.CacheControl;
import org.springframework.http.ResponseEntity;
import org.springframework.security.core.annotation.CurrentSecurityContext;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody; import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMapping;
@@ -19,6 +24,12 @@ import org.springframework.web.bind.annotation.RestController;
public class AuthController { public class AuthController {
private final AuthService authService; private final AuthService authService;
@GetMapping("/my-info")
public ResponseEntity<JwtUserClaimsDto> myInfo(
@CurrentSecurityContext(expression = "authentication.details") JwtUserClaimsDto user) {
return ResponseEntity.ok().cacheControl(CacheControl.noStore()).body(user);
}
@PostMapping("/login") @PostMapping("/login")
public LoginResponseDto login( public LoginResponseDto login(
@Valid @RequestBody LoginRequestDto request, RequestUserContext ctx) { @Valid @RequestBody LoginRequestDto request, RequestUserContext ctx) {

View File

@@ -14,6 +14,9 @@ import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema; import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponse; import io.swagger.v3.oas.annotations.responses.ApiResponse;
import jakarta.validation.Valid; import jakarta.validation.Valid;
import jakarta.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
@@ -93,13 +96,28 @@ public class ExportController {
@PostMapping( @PostMapping(
value = "/project/{projectType}/{pkey}/export/data", value = "/project/{projectType}/{pkey}/export/data",
produces = MediaType.APPLICATION_OCTET_STREAM_VALUE) produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
public ResponseEntity<byte[]> data( public void data(
@PathVariable ProjectType projectType, @PathVariable ProjectType projectType,
@PathVariable String pkey, @PathVariable String pkey,
@Valid @RequestBody RawDataExportRequestDto request, @Valid @RequestBody RawDataExportRequestDto request,
@Parameter(hidden = true) RequestUserContext ctx) { @Parameter(hidden = true) RequestUserContext ctx, HttpServletResponse response) throws IOException {
return download( ProjectRefDto project = new ProjectRefDto(projectType, pkey);
exportService.data(new ProjectRefDto(projectType, pkey), ctx.getUserRole(), request)); try (var file = exportService.prepareData(project, ctx.getUserRole(), request)) {
String fileName = FileUtil.extractFileNameFromPath(file.fileName())
.replaceAll("[\\p{Cntrl}\\\\/:*?\"<>|]", "_");
response.setContentType(MediaType.APPLICATION_OCTET_STREAM_VALUE);
response.setHeader(HttpHeaders.CONTENT_DISPOSITION,
ContentDisposition.attachment().filename(fileName, StandardCharsets.UTF_8).build().toString());
response.setHeader(HttpHeaders.CACHE_CONTROL, "no-store");
response.setContentLengthLong(Files.size(file.path()));
Files.copy(file.path(), response.getOutputStream());
} catch (IOException | RuntimeException error) {
// Before any file bytes are committed, preserve the normal JSON error response.
if (!response.isCommitted()) {
response.reset();
}
throw error;
}
} }
private ResponseEntity<byte[]> download(ExportFileDto file) { private ResponseEntity<byte[]> download(ExportFileDto file) {

View File

@@ -4,8 +4,10 @@ import com.kri.common.feature.web.BusinessException;
import com.kri.common.feature.web.RequestUserContext; import com.kri.common.feature.web.RequestUserContext;
import com.kri.kart.server.common.consts.KartConstant; import com.kri.kart.server.common.consts.KartConstant;
import com.kri.kart.server.dto.log.VisitLogRequestDto; import com.kri.kart.server.dto.log.VisitLogRequestDto;
import com.kri.kart.server.dto.project.ProjectRefDto;
import com.kri.kart.server.dto.share.ShareLinkDto; import com.kri.kart.server.dto.share.ShareLinkDto;
import com.kri.kart.server.service.LogService; import com.kri.kart.server.service.LogService;
import com.kri.kart.server.service.ProjectService;
import io.swagger.v3.oas.annotations.Parameter; import io.swagger.v3.oas.annotations.Parameter;
import jakarta.servlet.http.HttpServletRequest; import jakarta.servlet.http.HttpServletRequest;
import jakarta.validation.Valid; import jakarta.validation.Valid;
@@ -23,6 +25,7 @@ import org.springframework.web.bind.annotation.RestController;
@ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true") @ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true")
public class LogController { public class LogController {
private final LogService logService; private final LogService logService;
private final ProjectService projectService;
@PostMapping(value = "/visit", produces = MediaType.TEXT_PLAIN_VALUE) @PostMapping(value = "/visit", produces = MediaType.TEXT_PLAIN_VALUE)
public String visit( public String visit(
@@ -36,6 +39,12 @@ public class LogController {
|| !link.pkey().equals(request.pkey()))) { || !link.pkey().equals(request.pkey()))) {
throw BusinessException.forbidden("EXTERNAL_SCOPE_DENIED", "공유 프로젝트의 방문 로그만 기록할 수 있습니다."); throw BusinessException.forbidden("EXTERNAL_SCOPE_DENIED", "공유 프로젝트의 방문 로그만 기록할 수 있습니다.");
} }
if (link == null && (request.projectType() != null || request.pkey() != null)) {
if (request.projectType() == null || request.pkey() == null) {
throw BusinessException.badRequest("projectType과 pkey를 함께 입력해 주세요.");
}
projectService.requireAccess(new ProjectRefDto(request.projectType(), request.pkey()), ctx);
}
logService.writeVisitLog( logService.writeVisitLog(
request.projectType(), request.pkey(), request.logContent(), ctx); request.projectType(), request.pkey(), request.logContent(), ctx);
return "PASS"; return "PASS";

View File

@@ -8,6 +8,7 @@ import com.kri.kart.server.dto.project.ProgressDayDto;
import com.kri.kart.server.dto.project.ProgressScreenDto; import com.kri.kart.server.dto.project.ProgressScreenDto;
import com.kri.kart.server.dto.project.ProgressSearchDto; import com.kri.kart.server.dto.project.ProgressSearchDto;
import com.kri.kart.server.dto.project.ProgressTypeDto; import com.kri.kart.server.dto.project.ProgressTypeDto;
import com.kri.kart.server.dto.project.ProjectAccessDto;
import com.kri.kart.server.dto.project.ProjectLinkSearchDto; import com.kri.kart.server.dto.project.ProjectLinkSearchDto;
import com.kri.kart.server.dto.project.ProjectListDto; import com.kri.kart.server.dto.project.ProjectListDto;
import com.kri.kart.server.dto.project.ProjectRefDto; import com.kri.kart.server.dto.project.ProjectRefDto;
@@ -15,6 +16,7 @@ import com.kri.kart.server.dto.project.ProjectSummaryDto;
import com.kri.kart.server.service.CustomProgressService; import com.kri.kart.server.service.CustomProgressService;
import com.kri.kart.server.service.ProgressService; import com.kri.kart.server.service.ProgressService;
import com.kri.kart.server.service.ProjectService; import com.kri.kart.server.service.ProjectService;
import io.swagger.v3.oas.annotations.Operation;
import jakarta.validation.Valid; import jakarta.validation.Valid;
import java.util.List; import java.util.List;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
@@ -35,6 +37,14 @@ public class ProjectController {
private final ProgressService progressService; private final ProgressService progressService;
private final CustomProgressService customProgressService; private final CustomProgressService customProgressService;
@GetMapping("/{projectType}/{pkey}/access")
@Operation(summary = "프로젝트 접근 권한 확인", description = "접근 가능하면 allowed=true, 권한이 없으면 403을 반환합니다.")
public ProjectAccessDto access(
@PathVariable ProjectType projectType, @PathVariable String pkey, RequestUserContext ctx) {
// The project interceptor applies the same authorization as every project-scoped route.
return new ProjectAccessDto(true);
}
@GetMapping("/list") @GetMapping("/list")
public List<ProjectListDto> list( public List<ProjectListDto> list(
@Valid @ParameterObject @ModelAttribute ProjectLinkSearchDto search, RequestUserContext ctx) { @Valid @ParameterObject @ModelAttribute ProjectLinkSearchDto search, RequestUserContext ctx) {

View File

@@ -5,6 +5,7 @@ import com.kri.kart.server.common.enums.ProjectType;
import com.kri.kart.server.dto.project.ProjectRefDto; import com.kri.kart.server.dto.project.ProjectRefDto;
import com.kri.kart.server.dto.share.ShareLinkDto; import com.kri.kart.server.dto.share.ShareLinkDto;
import com.kri.kart.server.dto.share.ShareLinkRequestDto; import com.kri.kart.server.dto.share.ShareLinkRequestDto;
import com.kri.kart.server.service.ProjectService;
import com.kri.kart.server.service.ShareLinkService; import com.kri.kart.server.service.ShareLinkService;
import io.swagger.v3.oas.annotations.Operation; import io.swagger.v3.oas.annotations.Operation;
import jakarta.validation.Valid; import jakarta.validation.Valid;
@@ -21,6 +22,7 @@ import org.springframework.web.bind.annotation.RestController;
@ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true") @ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true")
public class ShareLinkController { public class ShareLinkController {
private final ShareLinkService shareLinkService; private final ShareLinkService shareLinkService;
private final ProjectService projectService;
@PostMapping("/api/v1/project/{projectType}/{pkey}/share") @PostMapping("/api/v1/project/{projectType}/{pkey}/share")
@Operation(summary = "공유 링크 발급 또는 재사용") @Operation(summary = "공유 링크 발급 또는 재사용")
@@ -34,7 +36,9 @@ public class ShareLinkController {
@GetMapping("/api/v1/share/{urlKey}") @GetMapping("/api/v1/share/{urlKey}")
@Operation(summary = "공유 링크 정보 조회", description = "내부 인증 필요. 랜딩 횟수를 증가시키지 않는다.") @Operation(summary = "공유 링크 정보 조회", description = "내부 인증 필요. 랜딩 횟수를 증가시키지 않는다.")
public ShareLinkDto lookup(@PathVariable String urlKey) { public ShareLinkDto lookup(@PathVariable String urlKey, RequestUserContext ctx) {
return shareLinkService.lookup(urlKey); ShareLinkDto link = shareLinkService.lookup(urlKey);
projectService.requireAccess(new ProjectRefDto(link.projectType(), link.pkey()), ctx);
return link;
} }
} }

View File

@@ -0,0 +1,13 @@
package com.kri.kart.server.dto.export;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
/** Completed download file owned by the caller until close. */
public record ExportTempFileDto(String fileName, Path path) implements AutoCloseable {
@Override
public void close() throws IOException {
Files.deleteIfExists(path);
}
}

View File

@@ -0,0 +1,6 @@
package com.kri.kart.server.dto.project;
import io.swagger.v3.oas.annotations.media.Schema;
@Schema(description = "프로젝트 접근 권한 확인 결과")
public record ProjectAccessDto(boolean allowed) {}

View File

@@ -0,0 +1,14 @@
package com.kri.kart.server.dto.quota;
import com.kri.kart.server.dto.statistics.StatisticsFilterDto;
import java.util.List;
import java.util.function.Function;
/** Internal quota interpretation; condition validation remains lazy to preserve failure order. */
public record PreparedQuotaDto(
String index, List<List<String>> banners, StatisticsFilterDto filter) {
public record Entry(Long target, String isUsed, Long quotaId, Function<String, String> value) {}
public record Condition(String dimension, String choice) {}
}

View File

@@ -3,10 +3,15 @@ package com.kri.kart.server.dto.statistics;
import lombok.Getter; import lombok.Getter;
import lombok.Setter; import lombok.Setter;
/**
* REPORT_NLP_WORD row. WORD_TAG STOP marks a 불용어 (stopword); a row with SOURCE_TEXT is a synonym
* that replaces SOURCE_TEXT with WORD_TEXT/WORD_TAG; any other row is a Kiwi user word.
*/
@Getter @Getter
@Setter @Setter
public class NlpDictionaryWordDto { public class NlpDictionaryWordDto {
private String wordText; private String wordText;
private String wordTag; private String wordTag;
private Integer wordScore; private Integer wordScore;
private String sourceText;
} }

View File

@@ -0,0 +1,14 @@
package com.kri.kart.server.dto.statistics;
import lombok.Getter;
import lombok.Setter;
/** Text-analysis replacement such as 엘지전자 → LG전자, with 1=ALL, 2=project, 3=question scope. */
@Getter
@Setter
public class NlpSynonymDto {
private String sourceText;
private String targetText;
private String targetTag;
private Integer scopeRank;
}

View File

@@ -1,6 +1,7 @@
package com.kri.kart.server.dto.statistics; package com.kri.kart.server.dto.statistics;
import com.fasterxml.jackson.annotation.JsonFormat; import com.fasterxml.jackson.annotation.JsonFormat;
import com.kri.kart.server.common.enums.WordMergeMode;
import io.swagger.v3.oas.annotations.media.Schema; import io.swagger.v3.oas.annotations.media.Schema;
import jakarta.validation.Valid; import jakarta.validation.Valid;
import jakarta.validation.constraints.Max; import jakarta.validation.constraints.Max;
@@ -55,4 +56,13 @@ public class StatisticsOptionsDto {
private boolean showNumericSum; private boolean showNumericSum;
@Valid @NotNull private List<StatisticsValueGroupDto> valueGroups = new ArrayList<>(); @Valid @NotNull private List<StatisticsValueGroupDto> valueGroups = new ArrayList<>();
@NotNull
@Schema(description = "텍스트 분석에서 같은 단어의 품사별 빈도 합산 방식. 생략하면 2(명사 계열만 합치기)")
private WordMergeMode wordMergeMode = WordMergeMode.NOUN;
@Min(1)
@Max(5)
@Schema(description = "텍스트 분석에서 공백 없이 붙은 명사를 합칠 최대 토큰 수. 1이면 합치지 않음. 생략하면 3")
private int compoundNounMaxTokens = 3;
} }

View File

@@ -1,7 +1,8 @@
package com.kri.kart.server.service.share; package com.kri.kart.server.interceptor;
import com.kri.common.feature.auth.dto.JwtUserClaimsDto;
import com.kri.common.feature.web.BusinessException; import com.kri.common.feature.web.BusinessException;
import com.kri.common.feature.web.RequestUserContext;
import com.kri.common.feature.web.RequestUserContextProvider;
import com.kri.kart.server.common.consts.KartConstant; import com.kri.kart.server.common.consts.KartConstant;
import com.kri.kart.server.dto.share.ShareLinkDto; import com.kri.kart.server.dto.share.ShareLinkDto;
import com.kri.kart.server.service.ShareLinkService; import com.kri.kart.server.service.ShareLinkService;
@@ -11,14 +12,13 @@ import java.net.URI;
import java.util.Map; import java.util.Map;
import java.util.Set; import java.util.Set;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.web.servlet.HandlerInterceptor; import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.HandlerMapping; import org.springframework.web.servlet.HandlerMapping;
/** 공용 인증이 검증한 identity와 REPORT_URL을 사용한다. 일반 사용자 역할 제한과는 별개다. */ /** 공용 인증이 검증한 identity와 REPORT_URL을 사용한다. 일반 사용자 역할 제한과는 별개다. */
@RequiredArgsConstructor @RequiredArgsConstructor
public class ExternalAccessInterceptor implements HandlerInterceptor { public class ExternalAccessInterceptor implements HandlerInterceptor {
private final RequestUserContextProvider requestUserContextProvider;
private final ShareLinkService shareLinkService; private final ShareLinkService shareLinkService;
@Override @Override
@@ -31,17 +31,12 @@ public class ExternalAccessInterceptor implements HandlerInterceptor {
// A new link or credential login may replace a previous external identity. // A new link or credential login may replace a previous external identity.
return true; return true;
} }
Authentication authentication = SecurityContextHolder.getContext().getAuthentication(); RequestUserContext ctx = requestUserContextProvider.resolve(request);
if (authentication == null String userId = ctx.getUserId();
|| !authentication.isAuthenticated()
|| !(authentication.getDetails() instanceof JwtUserClaimsDto claims)) {
return true;
}
String userId = claims.getUserId();
boolean externalRole = boolean externalRole =
Integer.valueOf(KartConstant.EXTERNAL_URL_COPY_ROLE).equals(claims.getUserRole()) Integer.valueOf(KartConstant.EXTERNAL_URL_COPY_ROLE).equals(ctx.getUserRole())
|| Integer.valueOf(KartConstant.EXTERNAL_PAGE_COPY_ROLE).equals(claims.getUserRole()); || Integer.valueOf(KartConstant.EXTERNAL_PAGE_COPY_ROLE).equals(ctx.getUserRole());
boolean externalTeam = KartConstant.EXTERNAL_TEAM.equals(claims.getUserTeam()); boolean externalTeam = KartConstant.EXTERNAL_TEAM.equals(ctx.getUserTeam());
boolean externalId = userId != null && userId.startsWith(KartConstant.EXTERNAL_USER_PREFIX); boolean externalId = userId != null && userId.startsWith(KartConstant.EXTERNAL_USER_PREFIX);
if (!externalRole && !externalTeam && !externalId) { if (!externalRole && !externalTeam && !externalId) {
return true; return true;
@@ -59,9 +54,12 @@ public class ExternalAccessInterceptor implements HandlerInterceptor {
link.pageOnly() link.pageOnly()
? KartConstant.EXTERNAL_PAGE_COPY_ROLE ? KartConstant.EXTERNAL_PAGE_COPY_ROLE
: KartConstant.EXTERNAL_URL_COPY_ROLE; : KartConstant.EXTERNAL_URL_COPY_ROLE;
if (!Integer.valueOf(expectedRole).equals(claims.getUserRole())) { if (!Integer.valueOf(expectedRole).equals(ctx.getUserRole())) {
throw denied(); throw denied();
} }
if ("GET".equals(request.getMethod()) && "/api/v1/auth/my-info".equals(pattern)) {
return true;
}
@SuppressWarnings("unchecked") @SuppressWarnings("unchecked")
Map<String, String> variables = Map<String, String> variables =
(Map<String, String>) request.getAttribute(HandlerMapping.URI_TEMPLATE_VARIABLES_ATTRIBUTE); (Map<String, String>) request.getAttribute(HandlerMapping.URI_TEMPLATE_VARIABLES_ATTRIBUTE);
@@ -76,6 +74,7 @@ public class ExternalAccessInterceptor implements HandlerInterceptor {
String page = URI.create(link.realUrl()).getPath(); String page = URI.create(link.realUrl()).getPath();
boolean allowed = "/api/v1/log/visit".equals(pattern); boolean allowed = "/api/v1/log/visit".equals(pattern);
if ("GET".equals(request.getMethod())) { if ("GET".equals(request.getMethod())) {
allowed |= pattern.equals("/api/v1/project/{projectType}/{pkey}/access");
allowed |= allowed |=
Set.of( Set.of(
"/api/v1/project/{projectType}/{pkey}/question/list", "/api/v1/project/{projectType}/{pkey}/question/list",

View File

@@ -0,0 +1,58 @@
package com.kri.kart.server.interceptor;
import com.kri.common.feature.web.BusinessException;
import com.kri.common.feature.web.RequestUserContext;
import com.kri.common.feature.web.RequestUserContextProvider;
import com.kri.kart.server.common.consts.KartConstant;
import com.kri.kart.server.common.enums.ProjectType;
import com.kri.kart.server.dto.project.ProjectRefDto;
import com.kri.kart.server.service.ProjectService;
import jakarta.servlet.DispatcherType;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import java.util.Map;
import lombok.RequiredArgsConstructor;
import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.HandlerMapping;
@RequiredArgsConstructor
public class ProjectAccessInterceptor implements HandlerInterceptor {
private final RequestUserContextProvider requestUserContextProvider;
private final ProjectService projectService;
@Override
public boolean preHandle(
HttpServletRequest request, HttpServletResponse response, Object handler) {
// ExternalAccessInterceptor has already checked the stored project/page/API scope.
if (request.getDispatcherType() == DispatcherType.ASYNC
|| request.getAttribute(KartConstant.REQUEST_LINK_ATTRIBUTE) != null) {
return true;
}
@SuppressWarnings("unchecked")
Map<String, String> variables =
(Map<String, String>) request.getAttribute(HandlerMapping.URI_TEMPLATE_VARIABLES_ATTRIBUTE);
if (variables == null
|| !variables.containsKey("projectType")
|| !variables.containsKey("pkey")) {
return true;
}
RequestUserContext ctx = requestUserContextProvider.resolve(request);
if (!ctx.isAuthenticated()) {
throw unauthenticated();
}
ProjectRefDto project;
try {
project = new ProjectRefDto(
ProjectType.fromCode(Integer.valueOf(variables.get("projectType"))),
variables.get("pkey"));
} catch (IllegalArgumentException exception) {
throw BusinessException.badRequest("projectType 또는 pkey가 올바르지 않습니다.");
}
projectService.requireAccess(project, ctx);
return true;
}
private BusinessException unauthenticated() {
return BusinessException.unauthorized("PROJECT_IDENTITY_REQUIRED", "프로젝트 접근 인증 정보가 필요합니다.");
}
}

View File

@@ -14,6 +14,8 @@ public interface DerivedMapper {
String lockProject(@Param("pkey") String pkey); String lockProject(@Param("pkey") String pkey);
boolean projectExists(@Param("pkey") String pkey);
List<DerivedDefinitionDto> definitions(@Param("pkey") String pkey); List<DerivedDefinitionDto> definitions(@Param("pkey") String pkey);
DerivedDefinitionDto find(@Param("pkey") String pkey, @Param("qid") String qid); DerivedDefinitionDto find(@Param("pkey") String pkey, @Param("qid") String qid);

View File

@@ -7,6 +7,7 @@ import com.kri.kart.server.dto.export.ExportTitleDto;
import java.time.LocalDate; import java.time.LocalDate;
import java.util.List; import java.util.List;
import org.apache.ibatis.annotations.Param; import org.apache.ibatis.annotations.Param;
import org.apache.ibatis.session.ResultHandler;
public interface ExportMapper { public interface ExportMapper {
String additionalColumns(@Param("pkey") String pkey); String additionalColumns(@Param("pkey") String pkey);
@@ -19,10 +20,11 @@ public interface ExportMapper {
String surveyType(@Param("pkey") String pkey); String surveyType(@Param("pkey") String pkey);
List<ExportDataDto> data( void data(
@Param("pkey") String pkey, @Param("pkey") String pkey,
@Param("type") String type, @Param("type") String type,
@Param("startDate") LocalDate startDate, @Param("startDate") LocalDate startDate,
@Param("endDate") LocalDate endDate, @Param("endDate") LocalDate endDate,
@Param("includeInterviewer") Boolean includeInterviewer); @Param("includeInterviewer") Boolean includeInterviewer,
ResultHandler<ExportDataDto> handler);
} }

View File

@@ -6,6 +6,12 @@ import java.util.List;
import org.apache.ibatis.annotations.Param; import org.apache.ibatis.annotations.Param;
public interface ProjectMapper { public interface ProjectMapper {
boolean hasAccess(
@Param("pkey") String pkey,
@Param("userRole") Integer userRole,
@Param("authenticatedUserTeam") String authenticatedUserTeam,
@Param("assignedPkeys") List<String> assignedPkeys);
List<ProjectListDto> list( List<ProjectListDto> list(
@Param("userRole") Integer userRole, @Param("userRole") Integer userRole,
@Param("authenticatedUserTeam") String authenticatedUserTeam, @Param("authenticatedUserTeam") String authenticatedUserTeam,

View File

@@ -27,6 +27,10 @@ public class DerivedRepository {
.lockProject(project.pkey()); .lockProject(project.pkey());
} }
public boolean projectExists(ProjectRefDto project) {
return surveySessionProvider.mapper(project, DerivedMapper.class).projectExists(project.pkey());
}
public List<DerivedDefinitionDto> definitions(ProjectRefDto project) { public List<DerivedDefinitionDto> definitions(ProjectRefDto project) {
return surveySessionProvider.mapper(project, DerivedMapper.class) return surveySessionProvider.mapper(project, DerivedMapper.class)
.definitions(project.pkey()); .definitions(project.pkey());

View File

@@ -9,6 +9,7 @@ import com.kri.kart.server.dto.project.ProjectRefDto;
import com.kri.kart.server.persistence.SurveySessionProvider; import com.kri.kart.server.persistence.SurveySessionProvider;
import com.kri.kart.server.persistence.mapper.ExportMapper; import com.kri.kart.server.persistence.mapper.ExportMapper;
import java.util.List; import java.util.List;
import java.util.function.Consumer;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.stereotype.Repository; import org.springframework.stereotype.Repository;
@@ -42,14 +43,14 @@ public class ExportRepository {
return surveySessionProvider.mapper(project, ExportMapper.class).surveyType(project.pkey()); return surveySessionProvider.mapper(project, ExportMapper.class).surveyType(project.pkey());
} }
public List<ExportDataDto> data( public void data(
ProjectRefDto project, RawDataExportRequestDto request, boolean includeInterviewer) { ProjectRefDto project, RawDataExportRequestDto request, boolean includeInterviewer, Consumer<ExportDataDto> consumer) {
return surveySessionProvider.mapper(project, ExportMapper.class) surveySessionProvider.mapper(project, ExportMapper.class)
.data( .data(
project.pkey(), project.pkey(),
request.getType(), request.getType(),
request.getStartDate(), request.getStartDate(),
request.getEndDate() == null ? request.getStartDate() : request.getEndDate(), request.getEndDate() == null ? request.getStartDate() : request.getEndDate(),
includeInterviewer); includeInterviewer, context -> consumer.accept(context.getResultObject()));
} }
} }

View File

@@ -2,6 +2,7 @@ package com.kri.kart.server.persistence.repository;
import com.kri.kart.server.common.enums.ProjectType; import com.kri.kart.server.common.enums.ProjectType;
import com.kri.kart.server.dto.project.ProjectListDto; import com.kri.kart.server.dto.project.ProjectListDto;
import com.kri.kart.server.dto.project.ProjectRefDto;
import com.kri.kart.server.dto.project.ProjectSummaryDto; import com.kri.kart.server.dto.project.ProjectSummaryDto;
import com.kri.kart.server.persistence.SurveySessionProvider; import com.kri.kart.server.persistence.SurveySessionProvider;
import com.kri.kart.server.persistence.mapper.ProjectMapper; import com.kri.kart.server.persistence.mapper.ProjectMapper;
@@ -24,6 +25,12 @@ public class ProjectRepository {
this.kartSqlSessionTemplate = kartSqlSessionTemplate; this.kartSqlSessionTemplate = kartSqlSessionTemplate;
} }
public boolean hasAccess(
ProjectRefDto project, Integer userRole, String userTeam, List<String> assignedPkeys) {
return surveySessionProvider.mapper(project, ProjectMapper.class)
.hasAccess(project.pkey(), userRole, userTeam, assignedPkeys);
}
public List<String> findMustViewPkeys(ProjectType type) { public List<String> findMustViewPkeys(ProjectType type) {
return kartSqlSessionTemplate.getMapper(ProjectMapper.class).mustViewPkeys(type.getCode()); return kartSqlSessionTemplate.getMapper(ProjectMapper.class).mustViewPkeys(type.getCode());
} }

View File

@@ -6,11 +6,15 @@ import com.kri.common.feature.web.BusinessException;
import com.kri.common.feature.web.RequestUserContext; import com.kri.common.feature.web.RequestUserContext;
import com.kri.kart.server.common.enums.ProjectType; import com.kri.kart.server.common.enums.ProjectType;
import com.kri.kart.server.dto.derived.DerivedCalculationDto; import com.kri.kart.server.dto.derived.DerivedCalculationDto;
import com.kri.kart.server.dto.derived.DerivedDataDto;
import com.kri.kart.server.dto.derived.DerivedDefinitionDto; import com.kri.kart.server.dto.derived.DerivedDefinitionDto;
import com.kri.kart.server.dto.derived.DerivedSaveRequestDto; import com.kri.kart.server.dto.derived.DerivedSaveRequestDto;
import com.kri.kart.server.dto.project.ProjectRefDto; import com.kri.kart.server.dto.project.ProjectRefDto;
import com.kri.kart.server.dto.question.QuestionDto;
import com.kri.kart.server.persistence.repository.DerivedRepository; import com.kri.kart.server.persistence.repository.DerivedRepository;
import com.kri.kart.server.service.derived.DerivedComputationService; import com.kri.kart.server.service.derived.DerivedComputationService;
import com.kri.kart.server.service.derived.DerivedRecalculationGuard;
import java.util.ArrayList;
import java.util.List; import java.util.List;
import java.util.Objects; import java.util.Objects;
import java.util.function.Supplier; import java.util.function.Supplier;
@@ -28,6 +32,7 @@ public class DerivedManagementService {
private final DerivedComputationService derivedComputationService; private final DerivedComputationService derivedComputationService;
private final LogService logService; private final LogService logService;
private final ObjectMapper objectMapper; private final ObjectMapper objectMapper;
private final DerivedRecalculationGuard derivedRecalculationGuard;
private final PlatformTransactionManager konTransactionManager; private final PlatformTransactionManager konTransactionManager;
private final PlatformTransactionManager konkiwiTransactionManager; private final PlatformTransactionManager konkiwiTransactionManager;
@@ -37,6 +42,7 @@ public class DerivedManagementService {
DerivedComputationService derivedComputationService, DerivedComputationService derivedComputationService,
LogService logService, LogService logService,
ObjectMapper objectMapper, ObjectMapper objectMapper,
DerivedRecalculationGuard derivedRecalculationGuard,
@Qualifier("konTransactionManager") PlatformTransactionManager konTransactionManager, @Qualifier("konTransactionManager") PlatformTransactionManager konTransactionManager,
@Qualifier("konkiwiTransactionManager") @Qualifier("konkiwiTransactionManager")
PlatformTransactionManager konkiwiTransactionManager) { PlatformTransactionManager konkiwiTransactionManager) {
@@ -45,6 +51,7 @@ public class DerivedManagementService {
this.derivedComputationService = derivedComputationService; this.derivedComputationService = derivedComputationService;
this.logService = logService; this.logService = logService;
this.objectMapper = objectMapper; this.objectMapper = objectMapper;
this.derivedRecalculationGuard = derivedRecalculationGuard;
this.konTransactionManager = konTransactionManager; this.konTransactionManager = konTransactionManager;
this.konkiwiTransactionManager = konkiwiTransactionManager; this.konkiwiTransactionManager = konkiwiTransactionManager;
} }
@@ -152,23 +159,62 @@ public class DerivedManagementService {
if (idkey == null || idkey.isBlank() || idkey.length() > 512) { if (idkey == null || idkey.isBlank() || idkey.length() > 512) {
throw BusinessException.badRequest("응답자 ID를 확인해 주세요."); throw BusinessException.badRequest("응답자 ID를 확인해 주세요.");
} }
return transaction( return derivedRecalculationGuard.execute(project, idkey, () -> {
project, if (!derivedRepository.projectExists(project)) {
() -> { throw BusinessException.notFound("PROJECT_NOT_FOUND", "프로젝트를 찾을 수 없습니다.");
requireProject(project); }
// Legacy creates an empty DERIVED_DATA row even if there is no eligible completed List<DerivedDefinitionDto> definitions = derivedRepository.definitions(project);
// response. List<String> derivedIds = derivedIds(definitions);
List<QuestionDto> questions = definitions.isEmpty() ? List.of() : questionService.list(project, 1001);
List<DerivedDataDto> data = definitions.isEmpty() ? List.of()
: derivedRepository.respondentResponses(project, idkey);
if (data.size() > 1) {
// Legacy joins can return multiple rows for one ID. Preserve their database reread
// semantics rather than assuming uniqueness or collapsing results in memory.
return transaction(project, () -> {
derivedRepository.ensureRespondent(project, idkey); derivedRepository.ensureRespondent(project, idkey);
var definitions = derivedRepository.definitions(project);
var derivedIds = derivedIds(definitions);
int matched = 0; int matched = 0;
for (var definition : definitions) { for (DerivedDefinitionDto definition : definitions) {
matched += calculate(project, definition, idkey, derivedIds); matched += calculate(project, definition, idkey, derivedIds, questions);
} }
return new DerivedCalculationDto(definitions.size(), matched); return new DerivedCalculationDto(definitions.size(), matched);
}); });
}
definitions.forEach(definition -> validateId(definition.getQid()));
var results = derivedComputationService.calculateRespondent(definitions, questions, data, derivedIds);
List<RespondentWrite> writes = new ArrayList<>();
int matched = 0;
for (int variable = 0; variable < definitions.size(); variable++) {
List<String> groups = new ArrayList<>();
for (var group : results.get(variable)) {
groups.add(group.isEmpty() ? null : JsonUtil.toJson(objectMapper, group));
if (!group.isEmpty()) {
matched++;
}
}
writes.add(new RespondentWrite(definitions.get(variable).getQid(), groups));
}
int matchedRespondents = matched;
return transaction(project, () -> {
// Keep the original empty-row contract even when there are no definitions or responses.
derivedRepository.ensureRespondent(project, idkey);
for (RespondentWrite write : writes) {
if (write.groups().isEmpty()) {
derivedRepository.clearRespondentData(project, write.qid(), idkey);
} else {
for (String group : write.groups()) {
// JSON_MODIFY updates only this variable on the current row; never replace DATJSON.
derivedRepository.saveData(project, write.qid(), idkey, group);
}
}
}
return new DerivedCalculationDto(definitions.size(), matchedRespondents);
});
});
} }
private record RespondentWrite(String qid, List<String> groups) {}
/** Stored derived variable IDs; uploaded response columns with these IDs are not imported. */ /** Stored derived variable IDs; uploaded response columns with these IDs are not imported. */
public List<String> derivedIds(ProjectRefDto project) { public List<String> derivedIds(ProjectRefDto project) {
return derivedIds(derivedRepository.definitions(project)); return derivedIds(derivedRepository.definitions(project));
@@ -201,6 +247,12 @@ public class DerivedManagementService {
DerivedDefinitionDto definition, DerivedDefinitionDto definition,
String idkey, String idkey,
List<String> derivedIds) { List<String> derivedIds) {
return calculate(project, definition, idkey, derivedIds, null);
}
private int calculate(
ProjectRefDto project, DerivedDefinitionDto definition, String idkey,
List<String> derivedIds, List<QuestionDto> questions) {
validateId(definition.getQid()); validateId(definition.getQid());
if (idkey == null) { if (idkey == null) {
derivedRepository.clearData(project, definition.getQid()); derivedRepository.clearData(project, definition.getQid());
@@ -213,7 +265,8 @@ public class DerivedManagementService {
: derivedRepository.respondentResponses(project, idkey); : derivedRepository.respondentResponses(project, idkey);
var results = var results =
derivedComputationService.calculate( derivedComputationService.calculate(
definition, questionService.list(project, idkey == null ? 0 : 1001), data, derivedIds); definition, questions == null ? questionService.list(project, idkey == null ? 0 : 1001) : questions,
data, derivedIds);
int matched = 0; int matched = 0;
for (int i = 0; i < data.size(); i++) { for (int i = 0; i < data.size(); i++) {
var group = results.get(i); var group = results.get(i);

View File

@@ -10,6 +10,7 @@ import com.kri.kart.server.common.consts.KartConstant;
import com.kri.kart.server.dto.export.ExportDataDto; import com.kri.kart.server.dto.export.ExportDataDto;
import com.kri.kart.server.dto.export.ExportDerivedHeaderDto; import com.kri.kart.server.dto.export.ExportDerivedHeaderDto;
import com.kri.kart.server.dto.export.ExportFileDto; import com.kri.kart.server.dto.export.ExportFileDto;
import com.kri.kart.server.dto.export.ExportTempFileDto;
import com.kri.kart.server.dto.export.ExportGuideDto; import com.kri.kart.server.dto.export.ExportGuideDto;
import com.kri.kart.server.dto.export.ExportHeaderDto; import com.kri.kart.server.dto.export.ExportHeaderDto;
import com.kri.kart.server.dto.export.ExportTitleDto; import com.kri.kart.server.dto.export.ExportTitleDto;
@@ -20,6 +21,7 @@ import com.kri.kart.server.persistence.repository.ExportRepository;
import com.kri.kart.server.persistence.repository.QuestionRepository; import com.kri.kart.server.persistence.repository.QuestionRepository;
import java.io.ByteArrayOutputStream; import java.io.ByteArrayOutputStream;
import java.io.IOException; import java.io.IOException;
import java.io.OutputStream;
import java.nio.file.Files; import java.nio.file.Files;
import java.nio.file.Path; import java.nio.file.Path;
import java.time.format.DateTimeFormatter; import java.time.format.DateTimeFormatter;
@@ -128,8 +130,26 @@ public class ExportService {
return rows; return rows;
} }
public ExportFileDto data( public ExportTempFileDto prepareData(
ProjectRefDto project, Integer userRole, RawDataExportRequestDto request) { ProjectRefDto project, Integer userRole, RawDataExportRequestDto request) throws IOException {
Path path = Files.createTempFile("kart-data-export-", ".xlsx");
try {
try (OutputStream output = Files.newOutputStream(path)) {
data(project, userRole, request, output);
}
return new ExportTempFileDto(dataFileName(project), path);
} catch (IOException | RuntimeException | Error error) {
try {
Files.deleteIfExists(path);
} catch (IOException cleanupError) {
error.addSuppressed(cleanupError);
}
throw error;
}
}
void data(
ProjectRefDto project, Integer userRole, RawDataExportRequestDto request, OutputStream output) throws IOException {
if (!List.of("COMPLETE", "INCOMPLETE").contains(Objects.toString(request.getType(), "")) if (!List.of("COMPLETE", "INCOMPLETE").contains(Objects.toString(request.getType(), ""))
|| (request.getStartDate() == null && request.getEndDate() != null) || (request.getStartDate() == null && request.getEndDate() != null)
|| (request.getStartDate() != null || (request.getStartDate() != null
@@ -138,37 +158,49 @@ public class ExportService {
throw BusinessException.badRequest("다운로드 유형과 날짜 범위를 확인해 주세요."); throw BusinessException.badRequest("다운로드 유형과 날짜 범위를 확인해 주세요.");
} }
Map<String, String> headers = headers(project, userRole); Map<String, String> headers = headers(project, userRole);
List<Map<String, String>> rows = new ArrayList<>();
rows.add(headers); // Row 1 contains storage keys, row 2 contains variable aliases.
boolean includeInterviewer = "2".equals(exportRepository.surveyType(project)); boolean includeInterviewer = "2".equals(exportRepository.surveyType(project));
for (ExportDataDto response : exportRepository.data(project, request, includeInterviewer)) { try (var workbook = ExcelMapper.createStreamingWorkbook(100)) {
String dat = response.getDat(); try (var writer = ExcelMapper.openStreamingMapWriter(workbook, new ArrayList<>(headers.keySet()), "DATA", 1)) {
if (dat == null || dat.isEmpty()) { writer.append(headers);
continue; // SQL STUFF returns NULL when all response groups are empty. exportRepository.data(project, request, includeInterviewer, response -> {
String dat = response.getDat();
if (dat == null || dat.isEmpty()) {
return;
}
JsonNode values = jsonObject(dat.replaceAll("\\\\U\\d*", ""));
Map<String, String> row = new LinkedHashMap<>();
for (String key : headers.keySet()) {
String value = responseValue(response, values, key, includeInterviewer);
if (value != null && !value.isEmpty()) {
row.put(key, value);
}
}
writer.append(row);
});
} }
// Preserve legacy ReplaceUnicodeEmpty after SQL UPPER/FOR XML serialization. // Metadata is bounded by the questionnaire; respondent rows are never accumulated.
JsonNode values = jsonObject(dat.replaceAll("\\\\U\\d*", "")); try (var writer = ExcelMapper.openStreamingMapWriter(workbook,
Map<String, String> row = new LinkedHashMap<>(); List.of("문항 타입", "문항 / 보기번호", "내용", "VALUE LABEL", "VARIABLE LABEL"), "GUIDE", 1)) {
for (String key : headers.keySet()) { for (ExportGuideDto guide : guideRows(project)) {
row.put(key, responseValue(response, values, key, includeInterviewer)); Map<String, String> row = new LinkedHashMap<>();
row.put("문항 타입", guide.questionType());
row.put("문항 / 보기번호", guide.variable());
row.put("내용", guide.content());
row.put("VALUE LABEL", guide.valueLabel());
row.put("VARIABLE LABEL", guide.variableLabel());
row.values().removeIf(value -> value == null || value.isEmpty());
writer.append(row);
}
} }
rows.add(row); workbook.write(output);
}
Workbook workbook = ExcelMapper.writeFromMapList(rows, "DATA");
try {
ExcelMapper.appendSheetToWorkbook(
workbook, guideRows(project), ExportGuideDto.class, "GUIDE", 1);
return file("DATA", project, workbook);
} catch (RuntimeException exception) {
try {
workbook.close();
} catch (IOException closeException) {
exception.addSuppressed(closeException);
}
throw exception;
} }
} }
public String dataFileName(ProjectRefDto project) {
return "[DATA]" + project.pkey() + "_"
+ DateTimeUtil.nowKstLocalDateTime().format(FILE_TIMESTAMP) + ".xlsx";
}
private String responseValue( private String responseValue(
ExportDataDto response, JsonNode values, String key, boolean includeInterviewer) { ExportDataDto response, JsonNode values, String key, boolean includeInterviewer) {
// Fixed SQL columns take precedence; IDKEY remains the value embedded in DAT. // Fixed SQL columns take precedence; IDKEY remains the value embedded in DAT.

View File

@@ -8,6 +8,7 @@ import com.kri.kart.server.common.enums.ProjectType;
import com.kri.kart.server.common.external.intranet.IntranetExtensionApiClient; import com.kri.kart.server.common.external.intranet.IntranetExtensionApiClient;
import com.kri.kart.server.dto.project.ProjectLinkSearchDto; import com.kri.kart.server.dto.project.ProjectLinkSearchDto;
import com.kri.kart.server.dto.project.ProjectListDto; import com.kri.kart.server.dto.project.ProjectListDto;
import com.kri.kart.server.dto.project.ProjectRefDto;
import com.kri.kart.server.dto.project.ProjectSummaryDto; import com.kri.kart.server.dto.project.ProjectSummaryDto;
import com.kri.kart.server.persistence.repository.ProjectRepository; import com.kri.kart.server.persistence.repository.ProjectRepository;
import java.util.ArrayList; import java.util.ArrayList;
@@ -112,6 +113,17 @@ public class ProjectService {
return selected; return selected;
} }
public void requireAccess(ProjectRefDto project, RequestUserContext ctx) {
requireIdentity(ctx);
if (StringUtil.normalizeBlank(ctx.getUserId()) == null) {
throw BusinessException.unauthorized("PROJECT_IDENTITY_REQUIRED", "프로젝트 접근 인증 정보가 부족합니다.");
}
if (!projectRepository.hasAccess(
project, ctx.getUserRole(), ctx.getUserTeam(), assignedPkeys(ctx))) {
throw BusinessException.forbidden("PROJECT_ACCESS_DENIED", "프로젝트에 접근할 권한이 없습니다.");
}
}
private void requireIdentity(RequestUserContext ctx) { private void requireIdentity(RequestUserContext ctx) {
if (ctx.getUserRole() == null if (ctx.getUserRole() == null
|| (!Integer.valueOf(1001).equals(ctx.getUserRole()) || (!Integer.valueOf(1001).equals(ctx.getUserRole())

View File

@@ -1,12 +1,10 @@
package com.kri.kart.server.service; package com.kri.kart.server.service;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.ObjectMapper;
import com.kri.common.feature.json.JsonUtil;
import com.kri.common.feature.util.StringUtil;
import com.kri.common.feature.web.BusinessException; import com.kri.common.feature.web.BusinessException;
import com.kri.common.feature.web.RequestUserContext; import com.kri.common.feature.web.RequestUserContext;
import com.kri.kart.server.dto.project.ProjectRefDto; import com.kri.kart.server.dto.project.ProjectRefDto;
import com.kri.kart.server.dto.quota.PreparedQuotaDto;
import com.kri.kart.server.dto.quota.QuotaResultDto; import com.kri.kart.server.dto.quota.QuotaResultDto;
import com.kri.kart.server.dto.quota.QuotaResultRequestDto; import com.kri.kart.server.dto.quota.QuotaResultRequestDto;
import com.kri.kart.server.dto.quota.QuotaSetDto; import com.kri.kart.server.dto.quota.QuotaSetDto;
@@ -15,14 +13,11 @@ import com.kri.kart.server.dto.statistics.StatisticsRequestDto;
import com.kri.kart.server.dto.statistics.StatisticsResponseDto; import com.kri.kart.server.dto.statistics.StatisticsResponseDto;
import com.kri.kart.server.persistence.repository.QuotaRepository; import com.kri.kart.server.persistence.repository.QuotaRepository;
import com.kri.kart.server.service.quota.QuotaComputationService; import com.kri.kart.server.service.quota.QuotaComputationService;
import com.kri.kart.server.service.statistics.LegacyFilterService; import com.kri.kart.server.service.quota.QuotaPreparationService;
import com.kri.kart.server.service.statistics.StatisticsAggregationService; import com.kri.kart.server.service.statistics.StatisticsAggregationService;
import com.kri.kart.server.service.statistics.StatisticsDimension; import com.kri.kart.server.service.statistics.StatisticsDimension;
import com.kri.kart.server.service.statistics.StatisticsDimensionService; import com.kri.kart.server.service.statistics.StatisticsDimensionService;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List; import java.util.List;
import java.util.Locale;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
@@ -35,7 +30,7 @@ public class QuotaService {
private final StatisticsService statisticsService; private final StatisticsService statisticsService;
private final StatisticsAggregationService statisticsAggregationService; private final StatisticsAggregationService statisticsAggregationService;
private final StatisticsDimensionService statisticsDimensionService; private final StatisticsDimensionService statisticsDimensionService;
private final LegacyFilterService legacyFilterService; private final QuotaPreparationService quotaPreparationService;
private final QuotaComputationService quotaComputationService; private final QuotaComputationService quotaComputationService;
private final LogService logService; private final LogService logService;
private final ObjectMapper objectMapper; private final ObjectMapper objectMapper;
@@ -57,44 +52,14 @@ public class QuotaService {
content.put("QSTITLE", quotaSet.getQsTitle()); content.put("QSTITLE", quotaSet.getQsTitle());
content.put("QSJSON", quotaSet.getQsJson()); content.put("QSJSON", quotaSet.getQsJson());
logService.writeSiteLog(project, "QUOTA VIEW", content, ctx); logService.writeSiteLog(project, "QUOTA VIEW", content, ctx);
JsonNode settings; PreparedQuotaDto settings =
try { quotaPreparationService.settings(quotaSet.getQsJson());
settings = JsonUtil.fromJson(objectMapper, quotaSet.getQsJson(), JsonNode.class); String index = settings.index();
} catch (IllegalArgumentException error) { List<List<String>> banners = settings.banners();
throw invalidSettings();
}
if (settings == null
|| !settings.isObject()
|| !settings.path("ROW").isArray()
|| !settings.path("COL").isArray()) {
throw invalidSettings();
}
String index =
settings.path("ROW").isEmpty() ? "" : dimension(settings.path("ROW").get(0), true);
List<String> columns = new ArrayList<>();
for (JsonNode column : settings.path("COL")) {
columns.add(dimension(column, false));
}
List<List<String>> banners =
columns.isEmpty()
? List.of()
: Arrays.stream(String.join("|", columns).split(",", -1))
.map(group -> Arrays.stream(group.split("\\|", -1)).map(String::trim).toList())
.toList();
if (banners.stream()
.anyMatch(
group ->
group.isEmpty() || group.size() > 5 || group.stream().anyMatch(String::isEmpty))) {
throw invalidSettings();
}
StatisticsRequestDto statisticsRequest = new StatisticsRequestDto(); StatisticsRequestDto statisticsRequest = new StatisticsRequestDto();
statisticsRequest.setIndex(index); statisticsRequest.setIndex(index);
statisticsRequest.setBanners(banners); statisticsRequest.setBanners(banners);
try { statisticsRequest.setFilter(settings.filter());
statisticsRequest.setFilter(legacyFilterService.parse(settings.get("FILTER")));
} catch (BusinessException exception) {
throw BusinessException.validationError("QUOTA_FILTER_INVALID", "저장된 쿼터 필터 형식을 확인해 주세요.");
}
statisticsRequest.setDataScope(request.getDataScope()); statisticsRequest.setDataScope(request.getDataScope());
statisticsRequest.setStartDate(request.getStartDate()); statisticsRequest.setStartDate(request.getStartDate());
statisticsRequest.setEndDate(request.getEndDate()); statisticsRequest.setEndDate(request.getEndDate());
@@ -105,7 +70,7 @@ public class QuotaService {
StatisticsDimension indexDimension = StatisticsDimension indexDimension =
statisticsDimensionService.resolve( statisticsDimensionService.resolve(
index, statisticsBatch.questions(), statisticsBatch.responses()); index, statisticsBatch.questions(), statisticsBatch.responses());
if ("text".equals(indexDimension.getQuestion().getQtype()) if ("text".equals(indexDimension.getQuestionType())
&& !indexDimension.isAttributeProjection()) { && !indexDimension.isAttributeProjection()) {
throw BusinessException.validationError( throw BusinessException.validationError(
"QUOTA_TEXT_UNSUPPORTED", "주관식 문자 문항은 쿼터 완료수 표로 집계할 수 없습니다."); "QUOTA_TEXT_UNSUPPORTED", "주관식 문자 문항은 쿼터 완료수 표로 집계할 수 없습니다.");
@@ -121,26 +86,8 @@ public class QuotaService {
matchIndex, matchIndex,
banners, banners,
table, table,
quotaRepository.quotaEntries(project, quotaOrder)); quotaPreparationService.entries(quotaRepository.quotaEntries(project, quotaOrder)));
return result; return result;
} }
private String dimension(JsonNode value, boolean allowEmpty) {
if (!value.isTextual()) {
throw invalidSettings();
}
String dimension = StringUtil.normalizeBlank(value.asText());
if (dimension == null) {
if (allowEmpty) {
return "";
}
throw invalidSettings();
}
return dimension.toUpperCase(Locale.ROOT);
}
private BusinessException invalidSettings() {
return BusinessException.validationError(
"QUOTA_CONFIGURATION_INVALID", "저장된 쿼터 ROW/COL 설정을 확인해 주세요.");
}
} }

View File

@@ -22,12 +22,10 @@ import com.kri.kart.server.service.statistics.StatisticsDataService;
import com.kri.kart.server.service.statistics.StatisticsDimension; import com.kri.kart.server.service.statistics.StatisticsDimension;
import com.kri.kart.server.service.statistics.StatisticsDimensionService; import com.kri.kart.server.service.statistics.StatisticsDimensionService;
import com.kri.kart.server.service.statistics.StatisticsFilterService; import com.kri.kart.server.service.statistics.StatisticsFilterService;
import com.kri.kart.server.service.statistics.StatisticsSourceService;
import com.kri.kart.server.service.statistics.StatisticsTextService; import com.kri.kart.server.service.statistics.StatisticsTextService;
import java.io.IOException; import java.io.IOException;
import java.util.ArrayList;
import java.util.List; import java.util.List;
import java.util.Locale;
import java.util.Objects;
import java.util.function.BooleanSupplier; import java.util.function.BooleanSupplier;
import java.util.function.Predicate; import java.util.function.Predicate;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
@@ -43,6 +41,7 @@ public class StatisticsService {
private final QuestionRepository questionRepository; private final QuestionRepository questionRepository;
private final QuestionService questionService; private final QuestionService questionService;
private final StatisticsDataService statisticsDataService; private final StatisticsDataService statisticsDataService;
private final StatisticsSourceService statisticsSourceService;
private final StatisticsDimensionService statisticsDimensionService; private final StatisticsDimensionService statisticsDimensionService;
private final StatisticsFilterService statisticsFilterService; private final StatisticsFilterService statisticsFilterService;
private final StatisticsAggregationService statisticsAggregationService; private final StatisticsAggregationService statisticsAggregationService;
@@ -188,7 +187,7 @@ public class StatisticsService {
needsProfileResponses(project, requests, sharedRequest, questions); needsProfileResponses(project, requests, sharedRequest, questions);
// Query the union of all storage roots once; calculate() narrows it again for each index. // Query the union of all storage roots once; calculate() narrows it again for each index.
List<String> storageRoots = List<String> storageRoots =
requests.stream().flatMap(request -> storageRoots(request).stream()).distinct().toList(); requests.stream().flatMap(request -> statisticsSourceService.storageRoots(request).stream()).distinct().toList();
List<StatisticsDataDto> data = List<StatisticsDataDto> data =
statisticsRepository.data(project, sharedRequest, includeProfileResponses, storageRoots); statisticsRepository.data(project, sharedRequest, includeProfileResponses, storageRoots);
return statisticsDataService.normalize(data).stream().filter(filter).toList(); return statisticsDataService.normalize(data).stream().filter(filter).toList();
@@ -203,8 +202,8 @@ public class StatisticsService {
if (project.projectType() != ProjectType.KIWI) { if (project.projectType() != ProjectType.KIWI) {
return false; return false;
} }
return requests.stream().anyMatch(request -> isProfileId(request.getIndex())) return requests.stream().anyMatch(request -> statisticsSourceService.isProfileId(request.getIndex()))
|| sharedRequest.getBanners().stream().flatMap(List::stream).anyMatch(this::isProfileId) || sharedRequest.getBanners().stream().flatMap(List::stream).anyMatch(statisticsSourceService::isProfileId)
|| filterUsesProfile(sharedRequest.getFilter(), questions); || filterUsesProfile(sharedRequest.getFilter(), questions);
} }
@@ -213,25 +212,17 @@ public class StatisticsService {
return false; return false;
} }
if (filter.getDimensionId() != null if (filter.getDimensionId() != null
&& isProfileId(statisticsFilterService.resolveQid(filter, questions))) { && statisticsSourceService.isProfileId(statisticsFilterService.resolveQid(filter, questions))) {
return true; return true;
} }
return filter.getChildren().stream().anyMatch(child -> filterUsesProfile(child, questions)); return filter.getChildren().stream().anyMatch(child -> filterUsesProfile(child, questions));
} }
private boolean isProfileId(String id) {
String normalized = StringUtil.normalizeBlank(id);
if (normalized == null) {
return false;
}
String upperCaseId = normalized.toUpperCase(Locale.ROOT);
return KartConstant.PROFILE_ID_PREFIXES.stream().anyMatch(upperCaseId::startsWith);
}
private StatisticsResponseDto calculate( private StatisticsResponseDto calculate(
StatisticsBatchDto statisticsBatch, StatisticsRequestDto request) { StatisticsBatchDto statisticsBatch, StatisticsRequestDto request) {
List<SurveyResponseDto> responses = List<SurveyResponseDto> responses =
withinStorageRoots(statisticsBatch.responses(), storageRoots(request)); statisticsSourceService.withinStorageRoots(
statisticsBatch.responses(), statisticsSourceService.storageRoots(request));
StatisticsDimension index = StatisticsDimension index =
statisticsDimensionService.resolve( statisticsDimensionService.resolve(
request.getIndex(), statisticsBatch.questions(), responses); request.getIndex(), statisticsBatch.questions(), responses);
@@ -242,34 +233,8 @@ public class StatisticsService {
request, statisticsBatch.questions(), responses); request, statisticsBatch.questions(), responses);
} }
/** The shared union query must not add respondents that a single-index query would omit. */
private List<SurveyResponseDto> withinStorageRoots(
List<SurveyResponseDto> responses, List<String> storageRoots) {
Predicate<String> storedUnderRoots =
key -> storageRoots.contains(key) || storageRoots.contains(key.split("_")[0]);
return responses.stream()
.filter(response -> response.values().keySet().stream().anyMatch(storedUnderRoots))
.toList();
}
/** Storage root keys selected by the original single-index query: index plus banners. */
private List<String> storageRoots(StatisticsRequestDto request) {
List<String> ids = new ArrayList<>();
ids.add(isStandard(request) ? request.getIndex() : KartConstant.TABULATION_STORAGE_ROOT);
request.getBanners().forEach(ids::addAll);
return ids.stream().map(this::toStorageRoot).distinct().toList();
}
/** e.g. "q1_2#a" -> "Q1": upper-case, drop the attribute projection and keep the root. */
private String toStorageRoot(String id) {
return Objects.toString(StringUtil.normalizeBlank(id), "")
.toUpperCase(Locale.ROOT)
.replace(KartConstant.ATTRIBUTE_PROJECTION_SUFFIX, "")
.split("_")[0];
}
private boolean isOpenText(StatisticsDimension index) { private boolean isOpenText(StatisticsDimension index) {
return "text".equals(index.getQuestion().getQtype()) && !index.isAttributeProjection(); return "text".equals(index.getQuestionType()) && !index.isAttributeProjection();
} }
private StatisticsResponseDto textAnalysis( private StatisticsResponseDto textAnalysis(
@@ -280,11 +245,17 @@ public class StatisticsService {
return StatisticsResponseDto.builder() return StatisticsResponseDto.builder()
.type("TEXT_ANALYSIS") .type("TEXT_ANALYSIS")
.questionId(index.getDimensionId()) .questionId(index.getDimensionId())
.title(index.getQuestion().getTitle()) .title(index.getTitle())
.columns(List.of()) .columns(List.of())
.headerRows(List.of()) .headerRows(List.of())
.rows(List.of()) .rows(List.of())
.words(statisticsTextService.analyze(project, index.getDimensionId(), responses)) .words(
statisticsTextService.analyze(
project,
index.getDimensionId(),
responses,
request.getWordMergeMode(),
request.getCompoundNounMaxTokens()))
.appliedOptions(request) .appliedOptions(request)
.build(); .build();
} }

View File

@@ -134,6 +134,37 @@ public class DerivedComputationService {
return results; return results;
} }
/** Reuse one respondent snapshot, feeding preceding variable results into later conditions. */
public List<List<ObjectNode>> calculateRespondent(
List<DerivedDefinitionDto> definitions, List<QuestionDto> questions,
List<DerivedDataDto> data, Collection<String> derivedIds) {
List<DerivedDataDto> working = new ArrayList<>();
for (DerivedDataDto row : data) {
DerivedDataDto copy = new DerivedDataDto();
copy.setIdkey(row.getIdkey());
copy.setDatjson(row.getDatjson());
copy.setDerivedJson(row.getDerivedJson());
working.add(copy);
}
List<List<ObjectNode>> prepared = new ArrayList<>();
for (DerivedDefinitionDto definition : definitions) {
List<ObjectNode> groups = calculate(definition, questions, working, derivedIds);
prepared.add(groups);
for (int row = 0; row < working.size(); row++) {
DerivedDataDto response = working.get(row);
ObjectNode derived = response.getDerivedJson() == null
? objectMapper.createObjectNode() : object(response.getDerivedJson());
if (groups.get(row).isEmpty()) {
derived.remove(definition.getQid());
} else {
derived.set(definition.getQid(), groups.get(row));
}
response.setDerivedJson(JsonUtil.toJson(objectMapper, derived));
}
}
return prepared;
}
private void flatten(ObjectNode grouped, Map<String, String> flat) { private void flatten(ObjectNode grouped, Map<String, String> flat) {
grouped grouped
.fields() .fields()

View File

@@ -0,0 +1,66 @@
package com.kri.kart.server.service.derived;
import com.kri.common.feature.web.BusinessException;
import com.kri.kart.server.dto.project.ProjectRefDto;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;
import java.util.function.Supplier;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Component;
/** Serialize duplicate respondent recalculations before reading their response snapshots. */
@Component
@ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true")
public class DerivedRecalculationGuard {
private final ConcurrentHashMap<RespondentKey, RespondentGate> respondents = new ConcurrentHashMap<>();
private final long waitNanos;
public DerivedRecalculationGuard() {
this(TimeUnit.SECONDS.toNanos(10));
}
DerivedRecalculationGuard(long waitNanos) {
this.waitNanos = waitNanos;
}
public <T> T execute(ProjectRefDto project, String idkey, Supplier<T> action) {
RespondentKey respondent = new RespondentKey(project, idkey);
RespondentGate gate = respondents.compute(respondent, (key, current) -> {
RespondentGate selected = current == null ? new RespondentGate() : current;
selected.users++;
return selected;
});
boolean acquired = false;
try {
acquired = gate.permit.tryAcquire(waitNanos, TimeUnit.NANOSECONDS);
if (!acquired) {
throw busy();
}
return action.get();
} catch (InterruptedException error) {
Thread.currentThread().interrupt();
throw busy();
} finally {
if (acquired) {
gate.permit.release();
}
respondents.computeIfPresent(respondent, (key, current) -> --current.users == 0 ? null : current);
}
}
private BusinessException busy() {
return BusinessException.of(
HttpStatus.SERVICE_UNAVAILABLE, "DERIVED_RECALCULATION_BUSY",
"파생변수 재계산 요청이 많습니다. 잠시 후 다시 요청해 주세요.");
}
private record RespondentKey(ProjectRefDto project, String idkey) {}
private static final class RespondentGate {
private final Semaphore permit = new Semaphore(1, true);
// Accessed only inside ConcurrentHashMap.compute/computeIfPresent for the respondent.
private int users;
}
}

View File

@@ -1,11 +1,9 @@
package com.kri.kart.server.service.quota; package com.kri.kart.server.service.quota;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kri.common.feature.json.JsonUtil;
import com.kri.common.feature.web.BusinessException; import com.kri.common.feature.web.BusinessException;
import com.kri.kart.server.dto.quota.PreparedQuotaDto.Condition;
import com.kri.kart.server.dto.quota.PreparedQuotaDto.Entry;
import com.kri.kart.server.dto.quota.QuotaCellDto; import com.kri.kart.server.dto.quota.QuotaCellDto;
import com.kri.kart.server.dto.quota.QuotaEntryDto;
import com.kri.kart.server.dto.quota.QuotaResultDto; import com.kri.kart.server.dto.quota.QuotaResultDto;
import com.kri.kart.server.dto.quota.QuotaRowDto; import com.kri.kart.server.dto.quota.QuotaRowDto;
import com.kri.kart.server.dto.quota.QuotaSetDto; import com.kri.kart.server.dto.quota.QuotaSetDto;
@@ -15,18 +13,19 @@ import com.kri.kart.server.dto.statistics.StatisticsRowDto;
import com.kri.kart.server.service.statistics.StatisticsArithmetic; import com.kri.kart.server.service.statistics.StatisticsArithmetic;
import com.kri.kart.server.service.statistics.StatisticsTableService; import com.kri.kart.server.service.statistics.StatisticsTableService;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.HashMap;
import java.util.List; import java.util.List;
import java.util.Map;
import java.util.function.Function;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
@Service @Service
@RequiredArgsConstructor @RequiredArgsConstructor
public class QuotaComputationService { public class QuotaComputationService {
private final ObjectMapper objectMapper; private final QuotaPreparationService quotaPreparationService;
private final StatisticsTableService statisticsTableService; private final StatisticsTableService statisticsTableService;
private record Entry(QuotaEntryDto quotaEntry, JsonNode values) {}
private record Target(long count, String isUsed, Long quotaId) {} private record Target(long count, String isUsed, Long quotaId) {}
public QuotaResultDto compute( public QuotaResultDto compute(
@@ -35,26 +34,11 @@ public class QuotaComputationService {
String matchIndex, String matchIndex,
List<List<String>> banners, List<List<String>> banners,
StatisticsResponseDto table, StatisticsResponseDto table,
List<QuotaEntryDto> quotaEntries) { List<Entry> entries) {
List<Entry> entries = Map<String, List<Condition>> conditions = new HashMap<>();
quotaEntries.stream() Function<String, List<Condition>> columnConditions =
.map( id -> conditions.computeIfAbsent(id, key -> quotaPreparationService.columnConditions(
quotaEntry -> { key, banners.isEmpty() ? List.of() : banners.get(0)));
JsonNode values;
try {
values =
JsonUtil.fromJson(objectMapper, quotaEntry.getQuotaJson(), JsonNode.class);
} catch (IllegalArgumentException error) {
throw BusinessException.validationError(
"QUOTA_DATA_INVALID", "저장된 쿼터 조건을 확인해 주세요.");
}
if (values == null || !values.isObject()) {
throw BusinessException.validationError(
"QUOTA_DATA_INVALID", "저장된 쿼터 조건을 확인해 주세요.");
}
return new Entry(quotaEntry, values);
})
.toList();
List<StatisticsColumnDto> columns = new ArrayList<>(); List<StatisticsColumnDto> columns = new ArrayList<>();
columns.add(table.columns().get(0)); columns.add(table.columns().get(0));
table.columns().stream() table.columns().stream()
@@ -99,7 +83,7 @@ public class QuotaComputationService {
} }
Target target = Target target =
target( target(
entries, row, matchIndex, columnId, banners.isEmpty() ? List.of() : banners.get(0)); entries, row, matchIndex, columnConditions, columnId);
rowTargetSum += target.count(); rowTargetSum += target.count();
cells.add(cell(columnIndex, complete, target)); cells.add(cell(columnIndex, complete, target));
} }
@@ -151,46 +135,30 @@ public class QuotaComputationService {
} }
private Target target( private Target target(
List<Entry> entries, StatisticsRowDto row, String index, String columnId, List<String> banners) { List<Entry> entries,
StatisticsRowDto row, String index,
Function<String, List<Condition>> columnConditions,
String columnId) {
List<Entry> matching = entries; List<Entry> matching = entries;
if (!index.isEmpty() if (!index.isEmpty()
&& StatisticsArithmetic.number(row.rowId()) != null && StatisticsArithmetic.number(row.rowId()) != null
&& !"TOTAL".equals(row.kind())) { && !"TOTAL".equals(row.kind())) {
matching = matching =
matching.stream() matching.stream()
.filter(entry -> value(entry.values(), index).equals(row.rowId())) .filter(entry -> entry.value().apply(index).equals(row.rowId()))
.toList(); .toList();
} }
if (!"i".equals(columnId)) { for (Condition condition : columnConditions.apply(columnId)) {
String[] values = columnId.split("_"); matching = matching.stream()
if (values.length - 1 > banners.size()) { .filter(entry -> entry.value().apply(condition.dimension()).equals(condition.choice())).toList();
throw BusinessException.validationError("QUOTA_COLUMNS_INVALID", "쿼터 배너와 집계 열이 일치하지 않습니다.");
}
for (int depth = 1; depth < values.length; depth++) {
String banner = banners.get(depth - 1);
String choice = values[depth];
matching =
matching.stream()
.filter(entry -> value(entry.values(), banner).equals(choice))
.toList();
}
} }
long count = long count =
matching.stream() matching.stream()
.map(Entry::quotaEntry) .mapToLong(entry -> entry.target() == null ? 0 : entry.target())
.mapToLong(entry -> entry.getOriCnt() == null ? 0 : entry.getOriCnt())
.sum(); .sum();
QuotaEntryDto only = matching.size() == 1 ? matching.get(0).quotaEntry() : null; Entry only = matching.size() == 1 ? matching.get(0) : null;
return new Target( return new Target(
count, only == null ? null : only.getIsUse(), only == null ? null : only.getId()); count, only == null ? null : only.isUsed(), only == null ? null : only.quotaId());
}
private String value(JsonNode values, String key) {
JsonNode value = values.get(key);
if (value == null || value.isNull() || !value.isValueNode()) {
throw BusinessException.validationError("QUOTA_DATA_INVALID", "쿼터 조건에 필요한 문항 값이 없습니다.");
}
return value.asText();
} }
private QuotaCellDto cell(int columnIndex, long complete, Target target) { private QuotaCellDto cell(int columnIndex, long complete, Target target) {

View File

@@ -0,0 +1,130 @@
package com.kri.kart.server.service.quota;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kri.common.feature.json.JsonUtil;
import com.kri.common.feature.util.StringUtil;
import com.kri.common.feature.web.BusinessException;
import com.kri.kart.server.dto.quota.PreparedQuotaDto;
import com.kri.kart.server.dto.quota.QuotaEntryDto;
import com.kri.kart.server.dto.statistics.StatisticsFilterDto;
import com.kri.kart.server.service.statistics.LegacyFilterService;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Locale;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service;
/** Interprets saved quota formats without changing target/error semantics. */
@Service
@RequiredArgsConstructor
public class QuotaPreparationService {
private final ObjectMapper objectMapper;
private final LegacyFilterService legacyFilterService;
public PreparedQuotaDto settings(String json) {
JsonNode settings;
try {
settings = JsonUtil.fromJson(objectMapper, json, JsonNode.class);
} catch (IllegalArgumentException error) {
throw invalidSettings();
}
if (settings == null
|| !settings.isObject()
|| !settings.path("ROW").isArray()
|| !settings.path("COL").isArray()) {
throw invalidSettings();
}
String index =
settings.path("ROW").isEmpty() ? "" : dimension(settings.path("ROW").get(0), true);
List<String> columns = new ArrayList<>();
for (JsonNode column : settings.path("COL")) {
columns.add(dimension(column, false));
}
List<List<String>> banners =
columns.isEmpty()
? List.of()
: Arrays.stream(String.join("|", columns).split(",", -1))
.map(group -> Arrays.stream(group.split("\\|", -1)).map(String::trim).toList())
.toList();
if (banners.stream()
.anyMatch(
group ->
group.isEmpty() || group.size() > 5 || group.stream().anyMatch(String::isEmpty))) {
throw invalidSettings();
}
StatisticsFilterDto filter;
try {
filter = legacyFilterService.parse(settings.get("FILTER"));
} catch (BusinessException exception) {
throw BusinessException.validationError("QUOTA_FILTER_INVALID", "저장된 쿼터 필터 형식을 확인해 주세요.");
}
return new PreparedQuotaDto(index, banners, filter);
}
public List<PreparedQuotaDto.Entry> entries(List<QuotaEntryDto> quotaEntries) {
return quotaEntries.stream()
.map(
quotaEntry -> {
JsonNode values;
try {
values =
JsonUtil.fromJson(objectMapper, quotaEntry.getQuotaJson(), JsonNode.class);
} catch (IllegalArgumentException error) {
throw BusinessException.validationError(
"QUOTA_DATA_INVALID", "저장된 쿼터 조건을 확인해 주세요.");
}
if (values == null || !values.isObject()) {
throw BusinessException.validationError(
"QUOTA_DATA_INVALID", "저장된 쿼터 조건을 확인해 주세요.");
}
return new PreparedQuotaDto.Entry(
quotaEntry.getOriCnt(), quotaEntry.getIsUse(), quotaEntry.getId(),
key -> value(values, key));
})
.toList();
}
public List<PreparedQuotaDto.Condition> columnConditions(String columnId, List<String> banners) {
if ("i".equals(columnId)) {
return List.of();
}
String[] values = columnId.split("_");
if (values.length - 1 > banners.size()) {
throw BusinessException.validationError("QUOTA_COLUMNS_INVALID", "쿼터 배너와 집계 열이 일치하지 않습니다.");
}
List<PreparedQuotaDto.Condition> conditions = new ArrayList<>();
for (int depth = 1; depth < values.length; depth++) {
conditions.add(new PreparedQuotaDto.Condition(banners.get(depth - 1), values[depth]));
}
return conditions;
}
private String value(JsonNode values, String key) {
JsonNode value = values.get(key);
if (value == null || value.isNull() || !value.isValueNode()) {
throw BusinessException.validationError("QUOTA_DATA_INVALID", "쿼터 조건에 필요한 문항 값이 없습니다.");
}
return value.asText();
}
private String dimension(JsonNode value, boolean allowEmpty) {
if (!value.isTextual()) {
throw invalidSettings();
}
String dimension = StringUtil.normalizeBlank(value.asText());
if (dimension == null) {
if (allowEmpty) {
return "";
}
throw invalidSettings();
}
return dimension.toUpperCase(Locale.ROOT);
}
private BusinessException invalidSettings() {
return BusinessException.validationError(
"QUOTA_CONFIGURATION_INVALID", "저장된 쿼터 ROW/COL 설정을 확인해 주세요.");
}
}

View File

@@ -1,22 +1,39 @@
package com.kri.kart.server.service.statistics; package com.kri.kart.server.service.statistics;
import com.kri.common.feature.util.StringUtil;
import com.kri.common.feature.web.BusinessException; import com.kri.common.feature.web.BusinessException;
import com.kri.kart.server.dto.statistics.NlpDictionaryWordDto; import com.kri.kart.server.dto.statistics.NlpDictionaryWordDto;
import com.kri.kart.server.dto.statistics.NlpTokenDto; import com.kri.kart.server.dto.statistics.NlpTokenDto;
import java.nio.file.Files; import java.nio.file.Files;
import java.nio.file.Path; import java.nio.file.Path;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashMap;
import java.util.HashSet;
import java.util.LinkedHashMap; import java.util.LinkedHashMap;
import java.util.List; import java.util.List;
import java.util.Locale;
import java.util.Map; import java.util.Map;
import java.util.Set;
import java.util.regex.Pattern;
import kr.pe.bab2min.Kiwi; import kr.pe.bab2min.Kiwi;
import kr.pe.bab2min.KiwiBuilder; import kr.pe.bab2min.KiwiBuilder;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpStatus; import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
@Slf4j
@Service @Service
public class KiwiNlpService { public class KiwiNlpService {
// Same as kiwipiepy's default typo cost threshold.
private static final float TYPO_COST_THRESHOLD = 2.5f;
private static final Set<String> COMPOUND_NOUN_TAGS = Set.of("NNG", "NNP", "SL");
private static final Set<String> NOUN_AFFIX_TAGS = Set.of("SL", "SH", "XPN", "XSN");
// Above every user dictionary score (10/50/100) so a synonym always wins.
private static final float SYNONYM_SCORE = 200f;
private static final Pattern WHITESPACE = Pattern.compile("\\s+");
private final Path modelDirectory; private final Path modelDirectory;
public KiwiNlpService( public KiwiNlpService(
@@ -24,34 +41,76 @@ public class KiwiNlpService {
this.modelDirectory = Path.of(modelDirectory); this.modelDirectory = Path.of(modelDirectory);
} }
/**
* Keeps the existing behavior: synonym rows (with SOURCE_TEXT) are ignored and there is no
* compound-noun or noun-affix joining.
*/
public List<NlpTokenDto> tokenize(List<String> texts, List<NlpDictionaryWordDto> dictionary) { public List<NlpTokenDto> tokenize(List<String> texts, List<NlpDictionaryWordDto> dictionary) {
List<NlpDictionaryWordDto> words =
dictionary.stream().filter(word -> !isSynonymRow(word)).toList();
return tokenize(texts, words, List.of(), 1);
}
/**
* Tokenizes texts with REPORT_NLP_WORD rows: a row with SOURCE_TEXT is a synonym that replaces
* SOURCE_TEXT with WORD_TEXT/WORD_TAG before anything else, any other row is a user word. When
* {@code compoundNounMaxTokens} is 2 or more, noun prefixes/suffixes are joined and adjacent
* nouns are merged into one compound noun (see {@link #joinCompoundNouns}).
*/
public List<NlpTokenDto> tokenize(
List<String> texts, List<NlpDictionaryWordDto> dictionary, int compoundNounMaxTokens) {
List<NlpDictionaryWordDto> words = new ArrayList<>();
List<NlpDictionaryWordDto> synonyms = new ArrayList<>();
for (NlpDictionaryWordDto word : dictionary) {
(isSynonymRow(word) ? synonyms : words).add(word);
}
return tokenize(texts, words, synonyms, compoundNounMaxTokens);
}
private static boolean isSynonymRow(NlpDictionaryWordDto word) {
return StringUtil.normalizeBlank(word.getSourceText()) != null;
}
private List<NlpTokenDto> tokenize(
List<String> texts,
List<NlpDictionaryWordDto> dictionary,
List<NlpDictionaryWordDto> synonyms,
int compoundNounMaxTokens) {
if (texts.isEmpty()) { if (texts.isEmpty()) {
return List.of(); return List.of();
} }
if (!Files.isRegularFile(modelDirectory.resolve("cong.mdl"))) { if (!Files.isRegularFile(modelDirectory.resolve("cong.mdl"))) {
throw unavailable(); throw unavailable();
} }
Map<String, Byte> tags = tags();
List<NlpTokenDto> result = new ArrayList<>(); List<NlpTokenDto> result = new ArrayList<>();
int options = KiwiBuilder.BuildOption.loadDefaultDict | KiwiBuilder.BuildOption.loadTypoDict; // Kiwi defaults: allomorph integration plus default, typo and multi-word dictionaries.
// Use the user-approved latest model; retain dictionary, allomorph and typo-processing rules. int options = KiwiBuilder.BuildOption.default_;
try (KiwiBuilder kiwiBuilder = try (KiwiBuilder kiwiBuilder =
new KiwiBuilder(modelDirectory.toString(), 1, options, KiwiBuilder.ModelType.cong)) { new KiwiBuilder(modelDirectory.toString(), 1, options, KiwiBuilder.ModelType.cong);
for (NlpDictionaryWordDto word : dictionary) { KiwiBuilder.PreparedTypoTransformer typoTransformer =
Byte tag = tags.get(word.getWordTag()); KiwiBuilder.basicTypoSetWithContinual.prepare()) {
if (tag != null && word.getWordText() != null && word.getWordScore() != null) { // User words and synonym targets are never merged with their neighbors.
try { Set<DictionaryWord> userWords = registerWords(kiwiBuilder, dictionary);
kiwiBuilder.addWord(word.getWordText(), tag, word.getWordScore()); List<Synonym> spacedSynonyms = registerSynonyms(kiwiBuilder, synonyms, userWords);
} catch (RuntimeException ignored) { boolean joinNouns = compoundNounMaxTokens > 1;
// Existing dictionary registration ignores an individually invalid word. // Recognize URL/email/hashtag/mention/serial spans, normalize informal codas,
} // analyze unknown words with the character model and correct common typos.
} int match = Kiwi.Match.allWithNormalizing | Kiwi.Match.oovChrModel;
if (joinNouns) {
match |= Kiwi.Match.joinNounPrefix | Kiwi.Match.joinNounSuffix;
} }
Kiwi.AnalyzeOption analyzeOption =
new Kiwi.AnalyzeOption(
match, null, (short) 0, 0f, typoTransformer, TYPO_COST_THRESHOLD);
try (Kiwi kiwi = kiwiBuilder.build()) { try (Kiwi kiwi = kiwiBuilder.build()) {
for (String text : texts) { for (String text : texts) {
for (Kiwi.Token token : kiwi.tokenize(text, new Kiwi.AnalyzeOption(0))) { List<Kiwi.Token> tokens =
String tag = Kiwi.POSTag.toString(token.tag); replaceSpacedSynonyms(
result.add(new NlpTokenDto(token.form, tag == null ? "UNK" : tag)); text, List.of(kiwi.tokenize(text, analyzeOption)), spacedSynonyms);
if (joinNouns) {
result.addAll(joinCompoundNouns(tokens, userWords, compoundNounMaxTokens));
} else {
tokens.forEach(token -> result.add(toToken(token)));
} }
} }
} }
@@ -62,6 +121,279 @@ public class KiwiNlpService {
return result; return result;
} }
/**
* Registers each word once with its highest scope score (10/50/100). Kiwi keeps the last score
* of a duplicate registration, and the dictionary query has no defined order.
*/
private Set<DictionaryWord> registerWords(
KiwiBuilder kiwiBuilder, List<NlpDictionaryWordDto> dictionary) {
Map<String, Byte> tags = tags();
Map<DictionaryWord, Integer> scores = new LinkedHashMap<>();
int skipped = 0;
for (NlpDictionaryWordDto word : dictionary) {
String text = StringUtil.normalizeBlank(word.getWordText());
String tagName = StringUtil.normalizeBlank(word.getWordTag());
Byte tag = tagName == null ? null : tags.get(tagName.toUpperCase(Locale.ROOT));
if (text == null || tag == null || word.getWordScore() == null) {
skipped++;
continue;
}
scores.merge(new DictionaryWord(text, tag), word.getWordScore(), Math::max);
}
int rejected = 0;
for (Map.Entry<DictionaryWord, Integer> entry : scores.entrySet()) {
try {
if (!kiwiBuilder.addWord(entry.getKey().text(), entry.getKey().tag(), entry.getValue())) {
rejected++;
}
} catch (Exception ignored) {
// The native call throws checked exceptions; an invalid word does not stop the analysis.
rejected++;
}
}
if (skipped > 0 || rejected > 0) {
int registered = scores.size() - rejected;
// Counts only: never log dictionary words.
log.warn(
"Kiwi user dictionary: registered={}, skipped={}, rejected={}",
registered,
skipped,
rejected);
}
return new HashSet<>(scores.keySet());
}
/**
* Registers synonym rows above every user word: SOURCE_TEXT is analyzed directly as
* WORD_TEXT/WORD_TAG, and a spaced source also registers its spaceless form. The most specific
* scope (score 100 > 50 > 10) wins for the same source, and an explicit spaceless source wins over
* a derived one. Returns spaced sources, longest first, for {@link #replaceSpacedSynonyms}; adds
* every target to {@code protectedWords}.
*/
private List<Synonym> registerSynonyms(
KiwiBuilder kiwiBuilder,
List<NlpDictionaryWordDto> synonyms,
Set<DictionaryWord> protectedWords) {
Map<String, Byte> tags = tags();
// Targets already registered as plain user words keep their scope score.
Set<DictionaryWord> registeredTargets = new HashSet<>(protectedWords);
Map<String, Synonym> sources = new LinkedHashMap<>();
Map<String, Integer> sourceRanks = new HashMap<>();
int skipped = 0;
for (NlpDictionaryWordDto synonym : synonyms) {
String source = normalizeSpaces(synonym.getSourceText());
String target = StringUtil.normalizeBlank(synonym.getWordText());
String tagName = StringUtil.normalizeBlank(synonym.getWordTag());
Byte tag = tagName == null ? null : tags.get(tagName.toUpperCase(Locale.ROOT));
if (source == null || target == null || tag == null) {
skipped++;
continue;
}
int rank = synonym.getWordScore() == null ? 0 : synonym.getWordScore();
if (!sources.containsKey(source) || rank > sourceRanks.get(source)) {
sources.put(source, new Synonym(source, target, tag));
sourceRanks.put(source, rank);
}
}
Map<String, Synonym> attached = new LinkedHashMap<>();
Map<String, Integer> attachedWeights = new HashMap<>();
List<Synonym> spaced = new ArrayList<>();
for (Synonym synonym : sources.values()) {
boolean explicit = !synonym.source().contains(" ");
String form = synonym.source().replace(" ", "");
int weight = sourceRanks.get(synonym.source()) * 2 + (explicit ? 1 : 0);
Integer current = attachedWeights.get(form);
if (current == null || weight > current) {
attached.put(form, new Synonym(form, synonym.target(), synonym.tag()));
attachedWeights.put(form, weight);
}
if (!explicit) {
spaced.add(synonym);
}
protectedWords.add(new DictionaryWord(synonym.target(), synonym.tag()));
}
int rejected = 0;
for (Synonym synonym : attached.values()) {
KiwiBuilder.AnalyzedMorph[] morphs = {
new KiwiBuilder.AnalyzedMorph(
synonym.target(), synonym.tag(), 0, synonym.source().length())
};
try {
// Kiwi requires the target morpheme to exist, e.g. "LG" is not in the default dictionary.
// Score 0 keeps a short target from being forced inside longer words.
if (registeredTargets.add(new DictionaryWord(synonym.target(), synonym.tag()))) {
kiwiBuilder.addWord(synonym.target(), synonym.tag(), 0f);
}
if (!kiwiBuilder.addPreAnalyzedWord(synonym.source(), morphs, SYNONYM_SCORE)) {
rejected++;
}
} catch (Exception ignored) {
// The native call throws checked exceptions; an invalid synonym does not stop the analysis.
rejected++;
}
}
if (skipped > 0 || rejected > 0) {
int registered = attached.size() - rejected;
// Counts only: never log synonym texts.
log.warn(
"Kiwi synonyms: registered={}, skipped={}, rejected={}",
registered,
skipped,
rejected);
}
spaced.sort(Comparator.comparingInt((Synonym synonym) -> synonym.source().length()).reversed());
return spaced;
}
/**
* Replaces a run of tokens whose original text, with spaces collapsed, equals a spaced synonym
* source. The run must start and end on token boundaries and must not continue into a noun of
* the same word, so "엘지 전자레인지" (엘지/전자/레인지) is not "엘지 전자".
*/
static List<Kiwi.Token> replaceSpacedSynonyms(
String text, List<Kiwi.Token> tokens, List<Synonym> spacedSynonyms) {
if (spacedSynonyms.isEmpty()) {
return tokens;
}
List<Kiwi.Token> result = new ArrayList<>();
int start = 0;
while (start < tokens.size()) {
int end = -1;
Synonym matched = null;
for (Synonym synonym : spacedSynonyms) {
end = matchEnd(text, tokens, start, synonym.source());
if (end > start && !continuesNoun(tokens, start, end)) {
matched = synonym;
break;
}
}
if (matched == null) {
result.add(tokens.get(start));
start++;
continue;
}
Kiwi.Token first = tokens.get(start);
Kiwi.Token last = tokens.get(end - 1);
Kiwi.Token replaced = new Kiwi.Token();
replaced.form = matched.target();
replaced.tag = matched.tag();
replaced.position = first.position;
replaced.length = (short) (last.position + last.length - first.position);
replaced.wordPosition = first.wordPosition;
replaced.sentPosition = first.sentPosition;
replaced.lineNumber = first.lineNumber;
result.add(replaced);
start = end;
}
return result;
}
/** Whether a noun is attached right before or after the run inside the same word. */
private static boolean continuesNoun(List<Kiwi.Token> tokens, int start, int end) {
boolean before =
start > 0
&& isAttached(tokens.get(start - 1), tokens.get(start))
&& isNounLike(tokens.get(start - 1));
boolean after =
end < tokens.size()
&& isAttached(tokens.get(end - 1), tokens.get(end))
&& isNounLike(tokens.get(end));
return before || after;
}
private static boolean isNounLike(Kiwi.Token token) {
String tag = tagName(token);
return tag.startsWith("N") || NOUN_AFFIX_TAGS.contains(tag);
}
/** Exclusive end index of the run starting at {@code start} that spells {@code source}, or -1. */
private static int matchEnd(String text, List<Kiwi.Token> tokens, int start, String source) {
int from = tokens.get(start).position;
for (int end = start; end < tokens.size(); end++) {
Kiwi.Token last = tokens.get(end);
int to = last.position + last.length;
if (from < 0 || to > text.length() || to < from) {
return -1;
}
String span = WHITESPACE.matcher(text.substring(from, to)).replaceAll(" ");
if (span.equals(source)) {
return end + 1;
}
if (span.length() >= source.length()) {
return -1;
}
}
return -1;
}
private static String normalizeSpaces(String raw) {
String normalized = StringUtil.normalizeBlank(raw);
return normalized == null ? null : WHITESPACE.matcher(normalized).replaceAll(" ");
}
/**
* Merges a run of 2..maxTokens NNG/NNP/SL tokens written without a space inside one word into a
* single noun, when the run contains a Korean noun. The merged tag is NNP if any part is NNP,
* otherwise NNG. User dictionary words always stay as they are, and longer runs are kept apart.
*/
static List<NlpTokenDto> joinCompoundNouns(
List<Kiwi.Token> tokens, Set<DictionaryWord> userWords, int maxTokens) {
List<NlpTokenDto> result = new ArrayList<>();
int start = 0;
while (start < tokens.size()) {
if (!isCompoundPart(tokens.get(start), userWords)) {
result.add(toToken(tokens.get(start)));
start++;
continue;
}
int end = start + 1;
while (end < tokens.size()
&& isCompoundPart(tokens.get(end), userWords)
&& isAttached(tokens.get(end - 1), tokens.get(end))) {
end++;
}
List<Kiwi.Token> run = tokens.subList(start, end);
boolean hasKoreanNoun = run.stream().anyMatch(token -> !isForeignWord(token));
if (run.size() > 1 && run.size() <= maxTokens && hasKoreanNoun) {
StringBuilder form = new StringBuilder();
boolean properNoun = false;
for (Kiwi.Token token : run) {
form.append(token.form);
properNoun |= "NNP".equals(tagName(token));
}
result.add(new NlpTokenDto(form.toString(), properNoun ? "NNP" : "NNG"));
} else {
run.forEach(token -> result.add(toToken(token)));
}
start = end;
}
return result;
}
private static boolean isCompoundPart(Kiwi.Token token, Set<DictionaryWord> userWords) {
return COMPOUND_NOUN_TAGS.contains(tagName(token))
&& !userWords.contains(new DictionaryWord(token.form, token.tag));
}
private static boolean isForeignWord(Kiwi.Token token) {
return "SL".equals(tagName(token));
}
/** Same word and no space or other character between the two tokens. */
private static boolean isAttached(Kiwi.Token previous, Kiwi.Token next) {
return previous.wordPosition == next.wordPosition
&& previous.position + previous.length == next.position;
}
private static NlpTokenDto toToken(Kiwi.Token token) {
return new NlpTokenDto(token.form, tagName(token));
}
private static String tagName(Kiwi.Token token) {
String tag = Kiwi.POSTag.toString(token.tag);
return tag == null ? "UNK" : tag;
}
private Map<String, Byte> tags() { private Map<String, Byte> tags() {
Map<String, Byte> result = new LinkedHashMap<>(); Map<String, Byte> result = new LinkedHashMap<>();
for (int code = Byte.MIN_VALUE; code <= Byte.MAX_VALUE; code++) { for (int code = Byte.MIN_VALUE; code <= Byte.MAX_VALUE; code++) {
@@ -77,4 +409,8 @@ public class KiwiNlpService {
return BusinessException.of( return BusinessException.of(
HttpStatus.SERVICE_UNAVAILABLE, "KIWI_NLP_UNAVAILABLE", "Kiwi NLP 라이브러리와 모델 설정을 확인해 주세요."); HttpStatus.SERVICE_UNAVAILABLE, "KIWI_NLP_UNAVAILABLE", "Kiwi NLP 라이브러리와 모델 설정을 확인해 주세요.");
} }
record DictionaryWord(String text, byte tag) {}
record Synonym(String source, String target, byte tag) {}
} }

View File

@@ -11,7 +11,6 @@ import com.kri.kart.server.dto.statistics.StatisticsRowDto;
import com.kri.kart.server.dto.statistics.StatisticsValueGroupDto; import com.kri.kart.server.dto.statistics.StatisticsValueGroupDto;
import com.kri.kart.server.dto.statistics.SurveyResponseDto; import com.kri.kart.server.dto.statistics.SurveyResponseDto;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections; import java.util.Collections;
import java.util.LinkedHashMap; import java.util.LinkedHashMap;
import java.util.LinkedHashSet; import java.util.LinkedHashSet;
@@ -19,6 +18,7 @@ import java.util.List;
import java.util.Locale; import java.util.Locale;
import java.util.Map; import java.util.Map;
import java.util.Objects; import java.util.Objects;
import java.util.function.Function;
import java.util.regex.Pattern; import java.util.regex.Pattern;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
@@ -53,9 +53,10 @@ public class StatisticsAggregationService {
StatisticsRequestDto request, StatisticsRequestDto request,
List<QuestionDto> questions, List<QuestionDto> questions,
List<SurveyResponseDto> responses) { List<SurveyResponseDto> responses) {
StatisticsDimension index = Function<String, StatisticsDimension> dimensions =
statisticsDimensionService.resolve(request.getIndex(), questions, responses); statisticsDimensionService.prepare(questions, responses);
List<Column> columns = columns(index, request, questions, responses); StatisticsDimension index = dimensions.apply(request.getIndex());
List<Column> columns = columns(index, request, responses, dimensions);
if (KartConstant.BANNER_CROSS_MODE.equals(request.getMode())) { if (KartConstant.BANNER_CROSS_MODE.equals(request.getMode())) {
return bannerCrossTable(index, columns, request); return bannerCrossTable(index, columns, request);
} }
@@ -65,7 +66,7 @@ public class StatisticsAggregationService {
addSummaryRows(rows, index, columns, request); addSummaryRows(rows, index, columns, request);
return statisticsTableService.table( return statisticsTableService.table(
index.getDimensionId(), index.getDimensionId(),
index.getQuestion().getTitle(), index.getTitle(),
columnHeaders(index, columns), columnHeaders(index, columns),
rows, rows,
request); request);
@@ -75,8 +76,8 @@ public class StatisticsAggregationService {
private List<Column> columns( private List<Column> columns(
StatisticsDimension index, StatisticsDimension index,
StatisticsRequestDto request, StatisticsRequestDto request,
List<QuestionDto> questions, List<SurveyResponseDto> responses,
List<SurveyResponseDto> responses) { Function<String, StatisticsDimension> preparedDimensions) {
List<SurveyResponseDto> total = statisticsSelectionService.total(responses, index); List<SurveyResponseDto> total = statisticsSelectionService.total(responses, index);
List<List<SurveyResponseDto>> cases = List<List<SurveyResponseDto>> cases =
index.getChoices().stream() index.getChoices().stream()
@@ -87,12 +88,12 @@ public class StatisticsAggregationService {
List<List<String>> banners = request.getBanners(); List<List<String>> banners = request.getBanners();
if (!index.getAttributes().isEmpty()) { if (!index.getAttributes().isEmpty()) {
// The old matrix summary replaces supplied banners with its own attribute projection. // The old matrix summary replaces supplied banners with its own attribute projection.
banners = List.of(List.of(index.getQuestion().getQid() + "#A")); banners = List.of(List.of(index.getAttributeBannerId()));
} }
for (int group = 0; group < banners.size(); group++) { for (int group = 0; group < banners.size(); group++) {
List<StatisticsDimension> dimensions = List<StatisticsDimension> dimensions =
banners.get(group).stream() banners.get(group).stream()
.map(id -> statisticsDimensionService.resolve(id, questions, responses)) .map(preparedDimensions)
.toList(); .toList();
if (dimensions.stream().anyMatch(this::isOpenAnswer)) { if (dimensions.stream().anyMatch(this::isOpenAnswer)) {
throw BusinessException.badRequest("주관식 문자·숫자 문항은 배너로 사용할 수 없습니다."); throw BusinessException.badRequest("주관식 문자·숫자 문항은 배너로 사용할 수 없습니다.");
@@ -123,7 +124,7 @@ public class StatisticsAggregationService {
for (StatisticsDimension.Choice choice : banner.getChoices()) { for (StatisticsDimension.Choice choice : banner.getChoices()) {
String columnId = parentId + "_" + choice.value(); String columnId = parentId + "_" + choice.value();
List<String> labels = new ArrayList<>(parentLabels); List<String> labels = new ArrayList<>(parentLabels);
labels.add(banner.getQuestion().getTitle()); labels.add(banner.getTitle());
labels.add(choice.label()); labels.add(choice.label());
List<SurveyResponseDto> bannerTotal = List<SurveyResponseDto> bannerTotal =
statisticsSelectionService.bannerTotal(total, banner, choice.value()); statisticsSelectionService.bannerTotal(total, banner, choice.value());
@@ -161,7 +162,7 @@ public class StatisticsAggregationService {
} }
private boolean isOpenAnswer(StatisticsDimension dimension) { private boolean isOpenAnswer(StatisticsDimension dimension) {
return KartConstant.OPEN_QUESTION_TYPES.contains(dimension.getQuestion().getQtype()) return KartConstant.OPEN_QUESTION_TYPES.contains(dimension.getQuestionType())
&& !dimension.isAttributeProjection(); && !dimension.isAttributeProjection();
} }
@@ -221,7 +222,7 @@ public class StatisticsAggregationService {
StatisticsDimension index, StatisticsDimension index,
List<Column> columns, List<Column> columns,
StatisticsRequestDto request) { StatisticsRequestDto request) {
String type = index.getQuestion().getQtype(); String type = index.getQuestionType();
if ("num".equals(type)) { if ("num".equals(type)) {
addStatistic(rows, columns, index, KartConstant.MEAN, request); addStatistic(rows, columns, index, KartConstant.MEAN, request);
for (String statistic : KartConstant.NUMERIC_STATISTICS) { for (String statistic : KartConstant.NUMERIC_STATISTICS) {
@@ -396,12 +397,11 @@ public class StatisticsAggregationService {
Column column, StatisticsDimension index, boolean totalColumn) { Column column, StatisticsDimension index, boolean totalColumn) {
List<SurveyResponseDto> source = List<SurveyResponseDto> source =
totalColumn ? column.total() : column.cases().stream().flatMap(List::stream).toList(); totalColumn ? column.total() : column.cases().stream().flatMap(List::stream).toList();
String baseKey = index.getBaseKey();
LinkedHashSet<String> keys = new LinkedHashSet<>(); LinkedHashSet<String> keys = new LinkedHashSet<>();
source.forEach( source.forEach(
response -> response ->
response.values().keySet().stream() response.values().keySet().stream()
.filter(key -> key.equals(baseKey) || key.startsWith(baseKey + "_")) .filter(index.getAnswerAccess().valueKeys())
.forEach(keys::add)); .forEach(keys::add));
List<Double> numbers = new ArrayList<>(); List<Double> numbers = new ArrayList<>();
for (String key : keys) { for (String key : keys) {
@@ -555,21 +555,15 @@ public class StatisticsAggregationService {
} }
private List<Double> missingValues(StatisticsDimension index) { private List<Double> missingValues(StatisticsDimension index) {
String values = index.getQuestion().getMissingValue(); return index.getMissingValues();
return values == null
? List.of()
: Arrays.stream(values.split(","))
.map(StatisticsArithmetic::number)
.filter(Objects::nonNull)
.toList();
} }
private boolean isReverse(StatisticsDimension index) { private boolean isReverse(StatisticsDimension index) {
return "1".equals(index.getQuestion().getIsReverse()); return index.isReverse();
} }
private boolean isNps(StatisticsDimension index) { private boolean isNps(StatisticsDimension index) {
return "1".equals(index.getQuestion().getIsNps()); return index.isNps();
} }
/** One row summing the cases of the given choice codes. */ /** One row summing the cases of the given choice codes. */
@@ -615,7 +609,7 @@ public class StatisticsAggregationService {
private List<StatisticsColumnDto> columnHeaders(StatisticsDimension index, List<Column> columns) { private List<StatisticsColumnDto> columnHeaders(StatisticsDimension index, List<Column> columns) {
List<StatisticsColumnDto> result = new ArrayList<>(); List<StatisticsColumnDto> result = new ArrayList<>();
result.add(new StatisticsColumnDto("LABEL", "LABEL", List.of(index.getQuestion().getTitle()))); result.add(new StatisticsColumnDto("LABEL", "LABEL", List.of(index.getTitle())));
for (Column column : columns) { for (Column column : columns) {
result.add(new StatisticsColumnDto(column.id(), columnKind(column), column.labels())); result.add(new StatisticsColumnDto(column.id(), columnKind(column), column.labels()));
} }

View File

@@ -0,0 +1,17 @@
package com.kri.kart.server.service.statistics;
import java.util.List;
import java.util.function.Predicate;
/** Prepared access to existing response maps; never owns or copies respondent values. */
public record StatisticsAnswerAccess(
boolean wholeSample, Predicate<String> valueKeys, Selection selection) {
@FunctionalInterface
public interface Selection {
List<Read> prepare(
String attribute, String multiValue, List<String> attributes, int numberedSlots);
}
public record Read(String key, Predicate<String> answeredKeys) {}
}

View File

@@ -1,21 +1,48 @@
package com.kri.kart.server.service.statistics; package com.kri.kart.server.service.statistics;
import com.kri.kart.server.dto.question.QuestionDto;
import java.util.List; import java.util.List;
import lombok.Getter; import lombok.Getter;
import lombok.RequiredArgsConstructor;
@Getter @Getter
@RequiredArgsConstructor
public class StatisticsDimension { public class StatisticsDimension {
private final String dimensionId; private final String dimensionId;
private final QuestionDto question;
private final String baseKey; private final String baseKey;
private final String attributeValue; private final String attributeValue;
private final List<String> attributes; private final List<String> attributes;
private final int rankCount; private final int rankCount;
private final boolean attributeProjection; private final boolean attributeProjection;
private final List<Choice> choices; private final List<Choice> choices;
private final String title;
private final String questionType;
private final String attributeBannerId;
private final List<Double> missingValues;
private final boolean reverse;
private final boolean nps;
private final StatisticsAnswerAccess answerAccess;
public StatisticsDimension(
String dimensionId, Definition definition, String baseKey, String attributeValue,
List<String> attributes, int rankCount, boolean attributeProjection, List<Choice> choices,
StatisticsAnswerAccess answerAccess) {
this.dimensionId = dimensionId;
this.baseKey = baseKey;
this.attributeValue = attributeValue;
this.attributes = attributes;
this.rankCount = rankCount;
this.attributeProjection = attributeProjection;
this.choices = choices;
this.title = definition.title();
this.questionType = definition.questionType();
this.attributeBannerId = definition.attributeBannerId();
this.missingValues = definition.missingValues();
this.reverse = definition.reverse();
this.nps = definition.nps();
this.answerAccess = answerAccess;
}
public record Definition(
String title, String questionType, String attributeBannerId,
List<Double> missingValues, boolean reverse, boolean nps) {}
public record Choice(String value, String label) {} public record Choice(String value, String label) {}

View File

@@ -9,10 +9,15 @@ import com.kri.kart.server.common.consts.KartConstant;
import com.kri.kart.server.dto.question.QuestionDto; import com.kri.kart.server.dto.question.QuestionDto;
import com.kri.kart.server.dto.statistics.SurveyResponseDto; import com.kri.kart.server.dto.statistics.SurveyResponseDto;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.LinkedHashSet; import java.util.LinkedHashSet;
import java.util.List; import java.util.List;
import java.util.Locale; import java.util.Locale;
import java.util.Map;
import java.util.Objects; import java.util.Objects;
import java.util.function.Function;
import java.util.function.Predicate;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
@@ -21,6 +26,7 @@ import org.springframework.stereotype.Service;
public class StatisticsDimensionService { public class StatisticsDimensionService {
private final ObjectMapper objectMapper; private final ObjectMapper objectMapper;
private final StatisticsSourceService statisticsSourceService;
/** /**
* Resolves an index or banner id. A blank id is the whole sample; "Q1#A" projects the attributes * Resolves an index or banner id. A blank id is the whole sample; "Q1#A" projects the attributes
@@ -73,7 +79,24 @@ public class StatisticsDimensionService {
List<StatisticsDimension.Choice> choices = choices(question, qid, projection, responses); List<StatisticsDimension.Choice> choices = choices(question, qid, projection, responses);
excludeScreenedOutChoices(question, qid, choices); excludeScreenedOutChoices(question, qid, choices);
return new StatisticsDimension( return new StatisticsDimension(
id, question, baseKey, attributeValue, attributes, rankCount, projection, choices); id, definition(question), baseKey, attributeValue, attributes, rankCount, projection,
choices, statisticsSourceService.access(baseKey));
}
/** Cache only within one calculation and its exact respondent population. */
public Function<String, StatisticsDimension> prepare(
List<QuestionDto> questions, List<SurveyResponseDto> responses) {
Map<String, StatisticsDimension> dimensions = new HashMap<>();
return id -> dimensions.computeIfAbsent(id, key -> resolve(key, questions, responses));
}
private StatisticsDimension.Definition definition(QuestionDto question) {
String missing = question.getMissingValue();
List<Double> missingValues = missing == null ? List.of() : Arrays.stream(missing.split(","))
.map(StatisticsArithmetic::number).filter(Objects::nonNull).toList();
return new StatisticsDimension.Definition(
question.getTitle(), question.getQtype(), question.getQid() + KartConstant.ATTRIBUTE_PROJECTION_SUFFIX,
missingValues, "1".equals(question.getIsReverse()), "1".equals(question.getIsNps()));
} }
public JsonNode metadata(String json) { public JsonNode metadata(String json) {
@@ -96,13 +119,13 @@ public class StatisticsDimensionService {
.build(); .build();
return new StatisticsDimension( return new StatisticsDimension(
"", "",
question, definition(question),
"", "",
"", "",
List.of(), List.of(),
0, 0,
false, false,
List.of(new StatisticsDimension.Choice("0", "전체"))); List.of(new StatisticsDimension.Choice("0", "전체")), statisticsSourceService.access(""));
} }
private List<StatisticsDimension.Choice> choices( private List<StatisticsDimension.Choice> choices(
@@ -126,11 +149,12 @@ public class StatisticsDimensionService {
/** Open questions have no label metadata; every distinct answer becomes a choice. */ /** Open questions have no label metadata; every distinct answer becomes a choice. */
private List<StatisticsDimension.Choice> answeredChoices( private List<StatisticsDimension.Choice> answeredChoices(
QuestionDto question, String qid, List<SurveyResponseDto> responses) { QuestionDto question, String qid, List<SurveyResponseDto> responses) {
Predicate<String> valueKeys = statisticsSourceService.valueKeys(qid);
LinkedHashSet<String> keys = new LinkedHashSet<>(); LinkedHashSet<String> keys = new LinkedHashSet<>();
responses.forEach( responses.forEach(
response -> response ->
response.values().keySet().stream() response.values().keySet().stream()
.filter(key -> key.equals(qid) || key.startsWith(qid + "_")) .filter(valueKeys)
.forEach(keys::add)); .forEach(keys::add));
LinkedHashSet<String> values = new LinkedHashSet<>(); LinkedHashSet<String> values = new LinkedHashSet<>();
for (String key : keys) { for (String key : keys) {

View File

@@ -8,16 +8,17 @@ import com.kri.kart.server.dto.question.QuestionDto;
import com.kri.kart.server.dto.statistics.StatisticsFilterDto; import com.kri.kart.server.dto.statistics.StatisticsFilterDto;
import com.kri.kart.server.dto.statistics.SurveyResponseDto; import com.kri.kart.server.dto.statistics.SurveyResponseDto;
import java.util.List; import java.util.List;
import java.util.Locale;
import java.util.Objects; import java.util.Objects;
import java.util.Set; import java.util.Set;
import java.util.function.Predicate; import java.util.function.Predicate;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.web.util.HtmlUtils;
/** Compiles a filter AST into a respondent predicate. User input is never turned into SQL. */ /** Compiles a filter AST into a respondent predicate. User input is never turned into SQL. */
@Service @Service
@RequiredArgsConstructor
public class StatisticsFilterService { public class StatisticsFilterService {
private final StatisticsSourceService statisticsSourceService;
public Predicate<SurveyResponseDto> compile( public Predicate<SurveyResponseDto> compile(
StatisticsFilterDto filter, List<QuestionDto> questions) { StatisticsFilterDto filter, List<QuestionDto> questions) {
@@ -25,29 +26,7 @@ public class StatisticsFilterService {
} }
public String resolveQid(StatisticsFilterDto filter, List<QuestionDto> questions) { public String resolveQid(StatisticsFilterDto filter, List<QuestionDto> questions) {
String dimension = StringUtil.normalizeBlank(filter.getDimensionId()); return statisticsSourceService.resolveFilterQid(filter, questions);
if (dimension == null
|| dimension.contains("#")
|| !Set.of("QID", "VARIABLE").contains(Objects.toString(filter.getReference(), ""))) {
throw BusinessException.badRequest("필터 문항과 참조 방식을 확인해 주세요.");
}
String id = dimension.toUpperCase(Locale.ROOT);
for (QuestionDto question : questions) {
String reference = reference(filter, question);
if (reference != null && reference.equalsIgnoreCase(id)) {
return question.getQid().toUpperCase(Locale.ROOT);
}
}
// Base rank/open IDs exist in saved data even though the UI lists expanded dimensions only.
String root = id.split("_")[0];
for (QuestionDto question : questions) {
String reference = reference(filter, question);
if (reference != null && reference.split("_")[0].equalsIgnoreCase(root)) {
return question.getQid().split("_")[0].toUpperCase(Locale.ROOT)
+ id.substring(root.length());
}
}
throw BusinessException.badRequest("필터 문항을 찾을 수 없습니다.");
} }
private Predicate<SurveyResponseDto> compile( private Predicate<SurveyResponseDto> compile(
@@ -94,11 +73,11 @@ public class StatisticsFilterService {
if (!hasValidValues(operator, filter.getValues())) { if (!hasValidValues(operator, filter.getValues())) {
throw BusinessException.badRequest("필터 비교 값의 개수나 유형이 올바르지 않습니다."); throw BusinessException.badRequest("필터 비교 값의 개수나 유형이 올바르지 않습니다.");
} }
String qid = resolveQid(filter, questions); Predicate<String> keys = statisticsSourceService.filterKeys(resolveQid(filter, questions));
Predicate<SurveyResponseDto> predicate = Predicate<SurveyResponseDto> predicate =
response -> response ->
response.filterValues().entrySet().stream() response.filterValues().entrySet().stream()
.filter(entry -> matchingKey(qid, entry.getKey())) .filter(entry -> keys.test(entry.getKey()))
.anyMatch(entry -> compare(entry.getValue(), operator, filter.getValues())); .anyMatch(entry -> compare(entry.getValue(), operator, filter.getValues()));
// Legacy negative multi conditions are NOT EXISTS of a matching value, including missing keys. // Legacy negative multi conditions are NOT EXISTS of a matching value, including missing keys.
return KartConstant.FILTER_NEGATED_OPERATORS.contains(operator) ? predicate.negate() : predicate; return KartConstant.FILTER_NEGATED_OPERATORS.contains(operator) ? predicate.negate() : predicate;
@@ -123,22 +102,6 @@ public class StatisticsFilterService {
&& !(value.isNumber() && !Double.isFinite(value.asDouble())); && !(value.isNumber() && !Double.isFinite(value.asDouble()));
} }
private String reference(StatisticsFilterDto filter, QuestionDto question) {
return "VARIABLE".equals(filter.getReference()) ? question.getVariable() : question.getQid();
}
/** Q1 matches Q1 and Q1_*; Q1_2 matches Q1_2 and Q1_*_2. */
private boolean matchingKey(String qid, String key) {
if (key.equals(qid)) {
return true;
}
String[] parts = qid.split("_");
if (parts.length == 1) {
return key.startsWith(qid + "_");
}
return parts.length == 2 && key.startsWith(parts[0] + "_") && key.endsWith("_" + parts[1]);
}
private boolean compare(String actual, String operator, List<JsonNode> operands) { private boolean compare(String actual, String operator, List<JsonNode> operands) {
if (Set.of("IS_ANSWERED", "IS_UNANSWERED").contains(operator)) { if (Set.of("IS_ANSWERED", "IS_UNANSWERED").contains(operator)) {
return actual != null; return actual != null;
@@ -172,8 +135,7 @@ public class StatisticsFilterService {
return number == null ? KartConstant.FILTER_NOT_COMPARABLE : Double.compare(number, expected.asDouble()); return number == null ? KartConstant.FILTER_NOT_COMPARABLE : Double.compare(number, expected.asDouble());
} }
// SQL Server's existing collation is case-insensitive; DAT also contains FOR XML escaping. // SQL Server's existing collation is case-insensitive; DAT also contains FOR XML escaping.
return HtmlUtils.htmlUnescape(actual) return statisticsSourceService.filterText(actual)
.replaceFirst(" +$", "")
.compareToIgnoreCase(expected.asText().replaceFirst(" +$", "")); .compareToIgnoreCase(expected.asText().replaceFirst(" +$", ""));
} }
} }

View File

@@ -17,7 +17,7 @@ public class StatisticsSelectionService {
List<SurveyResponseDto> source, StatisticsDimension dimension) { List<SurveyResponseDto> source, StatisticsDimension dimension) {
return select( return select(
source, source,
dimension.getBaseKey(), dimension.getAnswerAccess(),
true, true,
dimension.getAttributeValue(), dimension.getAttributeValue(),
"", "",
@@ -32,7 +32,7 @@ public class StatisticsSelectionService {
String multiValue = isMulti(dimension) ? value : ""; String multiValue = isMulti(dimension) ? value : "";
return select( return select(
source, source,
dimension.getBaseKey(), dimension.getAnswerAccess(),
false, false,
dimension.getAttributeValue(), dimension.getAttributeValue(),
multiValue, multiValue,
@@ -48,7 +48,7 @@ public class StatisticsSelectionService {
boolean projection = banner.isAttributeProjection(); boolean projection = banner.isAttributeProjection();
return select( return select(
source, source,
banner.getBaseKey(), banner.getAnswerAccess(),
multi || projection, multi || projection,
projection ? value : banner.getAttributeValue(), projection ? value : banner.getAttributeValue(),
multi && !projection ? value : "", multi && !projection ? value : "",
@@ -71,7 +71,7 @@ public class StatisticsSelectionService {
boolean projection = banner.isAttributeProjection(); boolean projection = banner.isAttributeProjection();
return select( return select(
source, source,
banner.getBaseKey(), banner.getAnswerAccess(),
false, false,
projection ? value : banner.getAttributeValue(), projection ? value : banner.getAttributeValue(),
multi && !projection ? value : "", multi && !projection ? value : "",
@@ -81,7 +81,7 @@ public class StatisticsSelectionService {
} }
private boolean isMulti(StatisticsDimension dimension) { private boolean isMulti(StatisticsDimension dimension) {
return "multi".equals(dimension.getQuestion().getQtype()); return "multi".equals(dimension.getQuestionType());
} }
/** /**
@@ -90,7 +90,7 @@ public class StatisticsSelectionService {
*/ */
private List<SurveyResponseDto> select( private List<SurveyResponseDto> select(
List<SurveyResponseDto> source, List<SurveyResponseDto> source,
String baseKey, StatisticsAnswerAccess access,
boolean anyAnswer, boolean anyAnswer,
String attributeValue, String attributeValue,
String multiValue, String multiValue,
@@ -98,21 +98,13 @@ public class StatisticsSelectionService {
List<String> attributes, List<String> attributes,
int numberedSlots) { int numberedSlots) {
List<SurveyResponseDto> result = new ArrayList<>(); List<SurveyResponseDto> result = new ArrayList<>();
int attributeCount = Math.max(1, attributes.size()); List<StatisticsAnswerAccess.Read> reads =
int slotCount = Math.max(1, numberedSlots); access.selection().prepare(attributeValue, multiValue, attributes, numberedSlots);
for (int attributeIndex = 0; attributeIndex < attributeCount; attributeIndex++) { for (StatisticsAnswerAccess.Read read : reads) {
String attribute = attributes.isEmpty() ? attributeValue : attributes.get(attributeIndex); for (SurveyResponseDto response : source) {
String attributeSuffix = attribute.isEmpty() ? "" : "_" + attribute; if (access.wholeSample() || matches(response.values(), read, anyAnswer, choiceValue)) {
for (int slot = 0; slot < slotCount; slot++) { // Preserve repeated attribute/slot occurrences and their original order.
String slotSuffix = result.add(response);
numberedSlots != 0 ? "_" + (slot + 1) : multiValue.isEmpty() ? "" : "_" + multiValue;
String suffix = slotSuffix + attributeSuffix;
for (SurveyResponseDto response : source) {
if (baseKey.isEmpty()
|| matches(response.values(), baseKey, suffix, anyAnswer, choiceValue)) {
// Legacy JArray.Merge concatenates; the same person can be counted in several slots.
result.add(response);
}
} }
} }
} }
@@ -120,24 +112,13 @@ public class StatisticsSelectionService {
} }
private boolean matches( private boolean matches(
Map<String, String> values, Map<String, String> values, StatisticsAnswerAccess.Read read,
String baseKey, boolean anyAnswer, String choiceValue) {
String suffix,
boolean anyAnswer,
String choiceValue) {
if (anyAnswer) { if (anyAnswer) {
return values.entrySet().stream() return values.entrySet().stream().anyMatch(entry ->
.anyMatch( read.answeredKeys().test(entry.getKey()) && !"".equals(entry.getValue()));
entry -> {
String key = entry.getKey();
boolean sameBase =
baseKey.contains("_")
? key.equals(baseKey) || key.startsWith(baseKey + "_")
: key.split("_")[0].equals(baseKey);
return sameBase && key.endsWith(suffix) && !"".equals(entry.getValue());
});
} }
Double actual = StatisticsArithmetic.number(values.get(baseKey + suffix)); Double actual = StatisticsArithmetic.number(values.get(read.key()));
Double expected = StatisticsArithmetic.number(choiceValue); Double expected = StatisticsArithmetic.number(choiceValue);
return actual != null && expected != null && actual.doubleValue() == expected.doubleValue(); return actual != null && expected != null && actual.doubleValue() == expected.doubleValue();
} }

View File

@@ -0,0 +1,126 @@
package com.kri.kart.server.service.statistics;
import com.kri.common.feature.util.StringUtil;
import com.kri.common.feature.web.BusinessException;
import com.kri.kart.server.common.consts.KartConstant;
import com.kri.kart.server.dto.question.QuestionDto;
import com.kri.kart.server.dto.statistics.StatisticsFilterDto;
import com.kri.kart.server.dto.statistics.StatisticsRequestDto;
import com.kri.kart.server.dto.statistics.SurveyResponseDto;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Objects;
import java.util.Set;
import java.util.function.Predicate;
import org.springframework.stereotype.Service;
import org.springframework.web.util.HtmlUtils;
/** KON/KIWI storage conventions, kept outside statistical calculations. */
@Service
public class StatisticsSourceService {
public StatisticsAnswerAccess access(String baseKey) {
Predicate<String> sameBase =
baseKey.contains("_")
? key -> key.equals(baseKey) || key.startsWith(baseKey + "_")
: key -> key.split("_")[0].equals(baseKey);
return new StatisticsAnswerAccess(
baseKey.isEmpty(),
valueKeys(baseKey),
(attributeValue, multiValue, attributes, numberedSlots) -> {
List<StatisticsAnswerAccess.Read> reads = new ArrayList<>();
int attributeCount = Math.max(1, attributes.size());
int slotCount = Math.max(1, numberedSlots);
for (int attributeIndex = 0; attributeIndex < attributeCount; attributeIndex++) {
String attribute = attributes.isEmpty() ? attributeValue : attributes.get(attributeIndex);
String attributeSuffix = attribute.isEmpty() ? "" : "_" + attribute;
for (int slot = 0; slot < slotCount; slot++) {
String slotSuffix = numberedSlots != 0
? "_" + (slot + 1) : multiValue.isEmpty() ? "" : "_" + multiValue;
String suffix = slotSuffix + attributeSuffix;
reads.add(new StatisticsAnswerAccess.Read(
baseKey + suffix, key -> sameBase.test(key) && key.endsWith(suffix)));
}
}
return reads;
});
}
public Predicate<String> valueKeys(String baseKey) {
String prefix = baseKey + "_";
return key -> key.equals(baseKey) || key.startsWith(prefix);
}
public Predicate<String> filterKeys(String qid) {
String[] parts = qid.split("_");
return key -> key.equals(qid)
|| (parts.length == 1 && key.startsWith(qid + "_"))
|| (parts.length == 2 && key.startsWith(parts[0] + "_") && key.endsWith("_" + parts[1]));
}
public String resolveFilterQid(StatisticsFilterDto filter, List<QuestionDto> questions) {
String dimension = StringUtil.normalizeBlank(filter.getDimensionId());
if (dimension == null || dimension.contains("#")
|| !Set.of("QID", "VARIABLE").contains(Objects.toString(filter.getReference(), ""))) {
throw BusinessException.badRequest("필터 문항과 참조 방식을 확인해 주세요.");
}
String id = dimension.toUpperCase(Locale.ROOT);
for (QuestionDto question : questions) {
String reference = reference(filter, question);
if (reference != null && reference.equalsIgnoreCase(id)) {
return question.getQid().toUpperCase(Locale.ROOT);
}
}
String root = id.split("_")[0];
for (QuestionDto question : questions) {
String reference = reference(filter, question);
if (reference != null && reference.split("_")[0].equalsIgnoreCase(root)) {
return question.getQid().split("_")[0].toUpperCase(Locale.ROOT)
+ id.substring(root.length());
}
}
throw BusinessException.badRequest("필터 문항을 찾을 수 없습니다.");
}
private String reference(StatisticsFilterDto filter, QuestionDto question) {
return "VARIABLE".equals(filter.getReference()) ? question.getVariable() : question.getQid();
}
public List<String> storageRoots(StatisticsRequestDto request) {
List<String> ids = new ArrayList<>();
ids.add(KartConstant.STANDARD_MODE.equals(request.getMode())
? request.getIndex() : KartConstant.TABULATION_STORAGE_ROOT);
request.getBanners().forEach(ids::addAll);
return ids.stream().map(this::storageRoot).distinct().toList();
}
private String storageRoot(String id) {
return Objects.toString(StringUtil.normalizeBlank(id), "")
.toUpperCase(Locale.ROOT).replace(KartConstant.ATTRIBUTE_PROJECTION_SUFFIX, "").split("_")[0];
}
public List<SurveyResponseDto> withinStorageRoots(
List<SurveyResponseDto> responses, List<String> roots) {
Predicate<String> stored = key -> roots.contains(key) || roots.contains(key.split("_")[0]);
return responses.stream()
.filter(response -> response.values().keySet().stream().anyMatch(stored)).toList();
}
public boolean isProfileId(String id) {
String normalized = StringUtil.normalizeBlank(id);
if (normalized == null) {
return false;
}
String upper = normalized.toUpperCase(Locale.ROOT);
return KartConstant.PROFILE_ID_PREFIXES.stream().anyMatch(upper::startsWith);
}
public String textAnswer(String value) {
return HtmlUtils.htmlUnescape(value.replace('¸', ','));
}
public String filterText(String value) {
return HtmlUtils.htmlUnescape(value).replaceFirst(" +$", "");
}
}

View File

@@ -1,22 +1,28 @@
package com.kri.kart.server.service.statistics; package com.kri.kart.server.service.statistics;
import com.kri.common.feature.util.StringUtil;
import com.kri.kart.server.common.consts.KartConstant; import com.kri.kart.server.common.consts.KartConstant;
import com.kri.kart.server.common.enums.WordMergeMode;
import com.kri.kart.server.dto.project.ProjectRefDto; import com.kri.kart.server.dto.project.ProjectRefDto;
import com.kri.kart.server.dto.statistics.NlpDictionaryWordDto;
import com.kri.kart.server.dto.statistics.NlpTokenDto; import com.kri.kart.server.dto.statistics.NlpTokenDto;
import com.kri.kart.server.dto.statistics.StatisticsWordDto; import com.kri.kart.server.dto.statistics.StatisticsWordDto;
import com.kri.kart.server.dto.statistics.SurveyResponseDto; import com.kri.kart.server.dto.statistics.SurveyResponseDto;
import com.kri.kart.server.persistence.repository.NlpRepository; import com.kri.kart.server.persistence.repository.NlpRepository;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.HashSet;
import java.util.LinkedHashMap; import java.util.LinkedHashMap;
import java.util.LinkedHashSet; import java.util.LinkedHashSet;
import java.util.List; import java.util.List;
import java.util.Locale;
import java.util.Map; import java.util.Map;
import java.util.Objects; import java.util.Objects;
import java.util.Set; import java.util.Set;
import java.util.function.Predicate;
import java.util.regex.Pattern;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.web.util.HtmlUtils;
/** Word frequencies of open text answers, using the Report DB NLP dictionary and tag scopes. */ /** Word frequencies of open text answers, using the Report DB NLP dictionary and tag scopes. */
@Service @Service
@@ -24,48 +30,147 @@ import org.springframework.web.util.HtmlUtils;
@ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true") @ConditionalOnProperty(prefix = "kri.kart.database", name = "enabled", havingValue = "true")
public class StatisticsTextService { public class StatisticsTextService {
// Noun family for WordMergeMode.NOUN: nouns, foreign words and Chinese characters.
private static final Set<String> NOUN_FAMILY_TAGS =
Set.of("NNG", "NNP", "NNB", "NR", "NP", "SL", "SH");
private static final Pattern WHITESPACE = Pattern.compile("\\s+");
private final NlpRepository nlpRepository; private final NlpRepository nlpRepository;
private final KiwiNlpService kiwiNlpService; private final KiwiNlpService kiwiNlpService;
private final StatisticsSourceService statisticsSourceService;
/**
* Keeps the existing behavior: no synonyms or stopwords, the same word with a different tag is
* counted separately and compound nouns are not merged.
*/
public List<StatisticsWordDto> analyze( public List<StatisticsWordDto> analyze(
ProjectRefDto project, String qid, List<SurveyResponseDto> responses) { ProjectRefDto project, String qid, List<SurveyResponseDto> responses) {
return analyze(project, qid, responses, false, WordMergeMode.NONE, 1);
}
/**
* Applies the REPORT_NLP_WORD synonyms (rows with SOURCE_TEXT), the stopwords (불용어, rows
* tagged STOP) and the given merge options.
*/
public List<StatisticsWordDto> analyze(
ProjectRefDto project,
String qid,
List<SurveyResponseDto> responses,
WordMergeMode mode,
int compoundNounMaxTokens) {
return analyze(project, qid, responses, true, mode, compoundNounMaxTokens);
}
private List<StatisticsWordDto> analyze(
ProjectRefDto project,
String qid,
List<SurveyResponseDto> responses,
boolean extended,
WordMergeMode mode,
int compoundNounMaxTokens) {
List<String> texts = answerTexts(qid, responses); List<String> texts = answerTexts(qid, responses);
Set<String> selectedTags = new LinkedHashSet<>(nlpRepository.tags(project.pkey(), qid)); Set<String> selectedTags = new LinkedHashSet<>(nlpRepository.tags(project.pkey(), qid));
// 불용어 (STOP) rows are not dictionary words: they are never registered in Kiwi.
List<NlpDictionaryWordDto> dictionary = new ArrayList<>();
Set<String> stopwords = new HashSet<>();
for (NlpDictionaryWordDto word : nlpRepository.words(project.pkey(), qid)) {
if (isStopwordRow(word)) {
String stopword = stopwordKey(word.getWordText());
if (stopword != null) {
stopwords.add(stopword);
}
} else {
dictionary.add(word);
}
}
// The original overload ignores synonym rows and does not merge compound nouns.
List<NlpTokenDto> tokens = List<NlpTokenDto> tokens =
kiwiNlpService.tokenize(texts, nlpRepository.words(project.pkey(), qid)); extended
// The same word with a different tag is counted separately, in first-appearance order. ? kiwiNlpService.tokenize(texts, dictionary, compoundNounMaxTokens)
: kiwiNlpService.tokenize(texts, dictionary);
// Count each word and tag pair in first-appearance order.
LinkedHashMap<NlpTokenDto, Long> counts = new LinkedHashMap<>(); LinkedHashMap<NlpTokenDto, Long> counts = new LinkedHashMap<>();
for (NlpTokenDto token : tokens) { for (NlpTokenDto token : tokens) {
if (isCounted(token, selectedTags)) { // Stopwords apply to the final word after synonyms and compound merging, for every tag.
boolean stopword = extended && stopwords.contains(stopwordKey(token.text()));
if (!stopword && isCounted(token, selectedTags)) {
counts.merge(token, 1L, Long::sum); counts.merge(token, 1L, Long::sum);
} }
} }
// Stable sort: equal counts keep their first-appearance order. // Stable sort: equal counts keep their first-appearance order.
return counts.entrySet().stream() return merge(counts, mode == null ? WordMergeMode.NONE : mode).entrySet().stream()
.sorted(Map.Entry.<NlpTokenDto, Long>comparingByValue().reversed()) .sorted(Map.Entry.<NlpTokenDto, Long>comparingByValue().reversed())
.limit(KartConstant.TOP_WORD_LIMIT) .limit(KartConstant.TOP_WORD_LIMIT)
.map(entry -> toWord(entry.getKey(), entry.getValue())) .map(entry -> toWord(entry.getKey(), entry.getValue()))
.toList(); .toList();
} }
/**
* Sums the same text across mergeable tags. The merged word keeps the tag with the highest count
* (first appearance on ties) and the position of its first appearance.
*/
private LinkedHashMap<NlpTokenDto, Long> merge(
LinkedHashMap<NlpTokenDto, Long> counts, WordMergeMode mode) {
if (mode == WordMergeMode.NONE) {
return counts;
}
LinkedHashMap<NlpTokenDto, LinkedHashMap<String, Long>> groups = new LinkedHashMap<>();
for (Map.Entry<NlpTokenDto, Long> entry : counts.entrySet()) {
NlpTokenDto token = entry.getKey();
boolean mergeable = mode == WordMergeMode.ALL || NOUN_FAMILY_TAGS.contains(token.tag());
NlpTokenDto key = mergeable ? new NlpTokenDto(token.text(), null) : token;
groups.computeIfAbsent(key, ignored -> new LinkedHashMap<>()).put(token.tag(), entry.getValue());
}
LinkedHashMap<NlpTokenDto, Long> result = new LinkedHashMap<>();
for (Map.Entry<NlpTokenDto, LinkedHashMap<String, Long>> group : groups.entrySet()) {
String tag = null;
long best = 0;
long total = 0;
for (Map.Entry<String, Long> tagCount : group.getValue().entrySet()) {
total += tagCount.getValue();
if (tag == null || tagCount.getValue() > best) {
tag = tagCount.getKey();
best = tagCount.getValue();
}
}
result.put(new NlpTokenDto(group.getKey().text(), tag), total);
}
return result;
}
/** 불용어 row: WORD_TAG is STOP, ignoring case and surrounding spaces. */
private boolean isStopwordRow(NlpDictionaryWordDto word) {
String tag = StringUtil.normalizeBlank(word.getWordTag());
return KartConstant.STOPWORD_TAG.equalsIgnoreCase(tag);
}
/** Stopword comparison key: trimmed, spaces collapsed and case-insensitive. */
private String stopwordKey(String text) {
String normalized = StringUtil.normalizeBlank(text);
return normalized == null
? null
: WHITESPACE.matcher(normalized).replaceAll(" ").toLowerCase(Locale.ROOT);
}
private StatisticsWordDto toWord(NlpTokenDto token, long count) { private StatisticsWordDto toWord(NlpTokenDto token, long count) {
return new StatisticsWordDto(token.text(), token.tag(), count); return new StatisticsWordDto(token.text(), token.tag(), count);
} }
/** Answers of the question and its sub-keys, grouped by key in first-appearance order. */ /** Answers of the question and its sub-keys, grouped by key in first-appearance order. */
private List<String> answerTexts(String qid, List<SurveyResponseDto> responses) { private List<String> answerTexts(String qid, List<SurveyResponseDto> responses) {
Predicate<String> valueKeys = statisticsSourceService.valueKeys(qid);
LinkedHashSet<String> keys = new LinkedHashSet<>(); LinkedHashSet<String> keys = new LinkedHashSet<>();
responses.forEach( responses.forEach(
response -> response ->
response.values().keySet().stream() response.values().keySet().stream()
.filter(key -> key.equals(qid) || key.startsWith(qid + "_")) .filter(valueKeys)
.forEach(keys::add)); .forEach(keys::add));
List<String> texts = new ArrayList<>(); List<String> texts = new ArrayList<>();
for (String key : keys) { for (String key : keys) {
responses.stream() responses.stream()
.map(response -> response.values().get(key)) .map(response -> response.values().get(key))
.filter(Objects::nonNull) .filter(Objects::nonNull)
.map(value -> HtmlUtils.htmlUnescape(value.replace('¸', ','))) .map(statisticsSourceService::textAnswer)
.forEach(texts::add); .forEach(texts::add);
} }
return texts; return texts;

View File

@@ -220,7 +220,7 @@ public class QuestionImportService {
info.put(textKey, label.get("label").asText() + "(기타)"); info.put(textKey, label.get("label").asText() + "(기타)");
} }
} }
if (itemType.equals("single")) { if (itemType.equals("single") || itemType.equals("rank")) {
for (var label : labels) { for (var label : labels) {
if (label.path("etc").asText().equals("1")) { if (label.path("etc").asText().equals("1")) {
String value = label.get("value").asText(); String value = label.get("value").asText();

View File

@@ -9,6 +9,8 @@ kri:
driver-class-name: com.mysql.cj.jdbc.Driver driver-class-name: com.mysql.cj.jdbc.Driver
kart: kart:
share:
public-origin: https://test-kart.kric.com
database: database:
konkiwi: konkiwi:
url: 'jdbc:sqlserver://mssql-7sq2n.vpc-cdb.ntruss.com;databaseName={konkiwi};encrypt=false' url: 'jdbc:sqlserver://mssql-7sq2n.vpc-cdb.ntruss.com;databaseName={konkiwi};encrypt=false'

View File

@@ -6,4 +6,8 @@ kri:
url: jdbc:mysql://db-md87m-kr.vpc-pub-cdb.ntruss.com:3306/kri_common?characterEncoding=UTF-8 url: jdbc:mysql://db-md87m-kr.vpc-pub-cdb.ntruss.com:3306/kri_common?characterEncoding=UTF-8
username: kriit username: kriit
password: it#51304 password: it#51304
driver-class-name: com.mysql.cj.jdbc.Driver driver-class-name: com.mysql.cj.jdbc.Driver
kart:
share:
public-origin: http://localhost:4014

View File

@@ -9,6 +9,8 @@ kri:
driver-class-name: com.mysql.cj.jdbc.Driver driver-class-name: com.mysql.cj.jdbc.Driver
kart: kart:
share:
public-origin: https://kart.kric.com
database: database:
konkiwi: konkiwi:
url: 'jdbc:sqlserver://mssql-7sq2n.vpc-cdb.ntruss.com;databaseName={konkiwi};encrypt=false' url: 'jdbc:sqlserver://mssql-7sq2n.vpc-cdb.ntruss.com;databaseName={konkiwi};encrypt=false'

View File

@@ -3,6 +3,9 @@ server:
forward-headers-strategy: framework forward-headers-strategy: framework
spring: spring:
datasource:
hikari:
leak-detection-threshold: 30000
application: application:
name: kart-server name: kart-server
profiles: profiles:

View File

@@ -36,6 +36,9 @@
<select id="lockProject" databaseId="survey" resultType="string"> <select id="lockProject" databaseId="survey" resultType="string">
SELECT PKEY FROM PROJECT WITH(UPDLOCK, HOLDLOCK) WHERE PKEY = #{pkey} SELECT PKEY FROM PROJECT WITH(UPDLOCK, HOLDLOCK) WHERE PKEY = #{pkey}
</select> </select>
<select id="projectExists" databaseId="survey" resultType="boolean">
SELECT CASE WHEN EXISTS (SELECT 1 FROM PROJECT WHERE PKEY = #{pkey}) THEN 1 ELSE 0 END
</select>
<select id="definitions" databaseId="survey" resultType="com.kri.kart.server.dto.derived.DerivedDefinitionDto"> <select id="definitions" databaseId="survey" resultType="com.kri.kart.server.dto.derived.DerivedDefinitionDto">
SELECT QID, QUESTIONJSON AS question_json, DERIVEDJSON AS derived_json, Qorder AS q_order SELECT QID, QUESTIONJSON AS question_json, DERIVEDJSON AS derived_json, Qorder AS q_order
FROM DERIVED_VARIABLE WHERE PKEY = #{pkey} ORDER BY Qorder FROM DERIVED_VARIABLE WHERE PKEY = #{pkey} ORDER BY Qorder
@@ -102,7 +105,7 @@
<![CDATA[ <![CDATA[
SELECT A.IDKEY, A.DATJSON, B.DATJSON AS derived_json FROM DATA A WITH(NOLOCK) SELECT A.IDKEY, A.DATJSON, B.DATJSON AS derived_json FROM DATA A WITH(NOLOCK)
LEFT JOIN (SELECT DISTINCT IDKEY, PKEY, DATJSON FROM DERIVED_DATA) B ON A.PKEY = B.PKEY AND A.IDKEY = B.IDKEY LEFT JOIN (SELECT DISTINCT IDKEY, PKEY, DATJSON FROM DERIVED_DATA) B ON A.PKEY = B.PKEY AND A.IDKEY = B.IDKEY
WHERE A.PKEY = #{pkey} AND A.IDKEY = #{idkey} AND A.Status = 10 AND A.SURVEY_TYPE <> 'TEST' WHERE A.PKEY = #{pkey} AND A.IDKEY = #{idkey} AND A.SURVEY_TYPE <> 'TEST'
]]> ]]>
</select> </select>
</mapper> </mapper>

View File

@@ -41,7 +41,7 @@
<select id="surveyType" databaseId="survey" resultType="string"> <select id="surveyType" databaseId="survey" resultType="string">
SELECT JSON_VALUE(PROJECTJSON, '$.must.surveytype') FROM PROJECT WHERE PKEY = #{pkey} SELECT JSON_VALUE(PROJECTJSON, '$.must.surveytype') FROM PROJECT WHERE PKEY = #{pkey}
</select> </select>
<select id="data" databaseId="survey" resultType="com.kri.kart.server.dto.export.ExportDataDto"> <select id="data" databaseId="survey" resultSetType="FORWARD_ONLY" fetchSize="100" useCache="false" resultType="com.kri.kart.server.dto.export.ExportDataDto">
<![CDATA[ <![CDATA[
SELECT IA.Status, DAT.DAT, DAT.start_date, DAT.end_date, DAT.interval_min SELECT IA.Status, DAT.DAT, DAT.start_date, DAT.end_date, DAT.interval_min
]]> ]]>

View File

@@ -11,13 +11,14 @@
) TAGS ) TAGS
</select> </select>
<select id="words" databaseId="kart" resultType="com.kri.kart.server.dto.statistics.NlpDictionaryWordDto"> <select id="words" databaseId="kart" resultType="com.kri.kart.server.dto.statistics.NlpDictionaryWordDto">
SELECT DISTINCT WORD_TAG, WORD_TEXT, WORD_SCORE FROM ( SELECT DISTINCT WORD_TAG, WORD_TEXT, SOURCE_TEXT, WORD_SCORE FROM (
SELECT WORD_TAG, WORD_TEXT, 10 AS WORD_SCORE FROM dbo.REPORT_NLP_WORD WHERE PKEY = 'ALL' SELECT WORD_TAG, WORD_TEXT, SOURCE_TEXT, 10 AS WORD_SCORE FROM dbo.REPORT_NLP_WORD
WHERE PKEY = 'ALL'
UNION UNION
SELECT WORD_TAG, WORD_TEXT, 50 AS WORD_SCORE FROM dbo.REPORT_NLP_WORD SELECT WORD_TAG, WORD_TEXT, SOURCE_TEXT, 50 AS WORD_SCORE FROM dbo.REPORT_NLP_WORD
WHERE PKEY = #{pkey} AND QID = 'ALL' WHERE PKEY = #{pkey} AND QID = 'ALL'
UNION UNION
SELECT WORD_TAG, WORD_TEXT, 100 AS WORD_SCORE FROM dbo.REPORT_NLP_WORD SELECT WORD_TAG, WORD_TEXT, SOURCE_TEXT, 100 AS WORD_SCORE FROM dbo.REPORT_NLP_WORD
WHERE PKEY = #{pkey} AND QID = #{qid} WHERE PKEY = #{pkey} AND QID = #{qid}
) WORDS ) WORDS
</select> </select>

View File

@@ -39,6 +39,12 @@
</if> </if>
<include refid="assignmentVisibility"/> <include refid="assignmentVisibility"/>
</sql> </sql>
<select id="hasAccess" databaseId="survey" resultType="boolean">
SELECT CASE WHEN EXISTS (
SELECT 1 FROM PROJECT WHERE PKEY = #{pkey}
<include refid="listVisibility"/>
) THEN 1 ELSE 0 END
</select>
<select id="list" databaseId="survey" resultType="com.kri.kart.server.dto.project.ProjectListDto"> <select id="list" databaseId="survey" resultType="com.kri.kart.server.dto.project.ProjectListDto">
SELECT PKEY, <if test="projectType == 1">'KIWI - ' + </if> SELECT PKEY, <if test="projectType == 1">'KIWI - ' + </if>
<![CDATA[ <![CDATA[