Release Notes for Helix Core Search Version 2023.1 Introduction HelixCore Search is a REST service providing an API and simple web application to support other Helix applications such as Swarm and DAM. It facilitates indexing the contents of your Helix Server and searching Helix meta-data and file content. HelixCore Search needs a connection to ElasticSearch and a HelixCore server. Perforce numbers releases YYYY.R/CCCCCC, for example, 2012.1/40716. YYYY is the year; R is the release of that year; CCCCCC is the bug fix change level. Each bug fix in these release notes is marked by its change number. Any build includes (1) all bug fixes of all previous releases and (2) all bug fixes of the current release up to the bug fix change level. Please send all feedback to support@perforce.com. Minimum Requirements Elasticsearch 7.14.0 - 8.6.0 Helix Core server 2020.1 or later (2021.2 is required for Windows Extension support) Supported Platforms Windows (x86_64) Windows 2016 server Windows 2019 server Windows 2022 server Linux (x86_64) Ubuntu 18.04 Ubuntu 20.04 Ubuntu 22.04 CentOS 7 Rocky Linux 8 Rocky Linux 9 Documentation Please see the official documentation for installation instructions and usage examples. https://www.perforce.com/manuals/p4search/Content/P4Search/Home-p4search.html ------------------------------------------- Updates in 2023.1 Patch 1 (2023.1/2423229) (2023/03/28) #2422850 (Job job114929 job115033) Fix Unauthorized error when trying to match serverId (ip address) with authTickets key containing hostName in it. ------------------------------------------- Updates in 2023.1 (2023.1/2418278) (2023/03/15) #2414296 2414244 (Job job114807) Enhanced bulk-indexing customAttributes to use Java native types. #2417394 2417422 (Job job114806) Elastic RequestException 'Request Entity Too Large' causes remaining files to fail the index. #2416606 (Job job114768) Swagger docs for API endpoints. #2417056 (Job job114749) Doc update for Helix Search API endpoints #2409857 2417081 (Job job114526) Find texture files for OBJ and FBX files for rendering. #2401865 2404914 (Job job113492) Auto update extensions on startup. #2398260 (Job job113491) Update extension using out of date status. #2412195 2412195 (Job job114028) DeepDetect: add support for GoogleNet ------------------------------------------- Bug fixes in 2023.1 (2023.1/2418278) (2023/03/15) #2417106 2417421 2417422 (Job job114764) Large OBJ (vector file with text content) fails content index in ES. #2416455 (Job job114735) CORS error when windows user performs "Try it out" in swagger api endpoint. #2415832 2416693 2417496 (Job job114728) PSD library error causes indexer to crash and fails to generate subsequent thumbnails. #2405511 (Job job114345) Thumbnail not generated when filename has '+' or other reserved characters. ------------------------------------------- Updates in 2022.3 Patch 2 (2022.3/2391710) (2023/01/16) #2391238 (Job job113889) Update P4Java 2022.1.2390907. Resolves file corruption for compressed text+C files over 64K. #2382274 (Job job113360, job113362, job113670) Avoid duplicate calls to translations for messages that throw Exceptions. #2374433 (Job job113652) Add UUID to F3D temp file name to avoid concurrency issues. #2376837 (Job job113552) Removed duplicate TIFF processing library. ------------------------------------------- Updates in 2022.3 Patch 1 (2022.3/2372976) (2022/11/18) #2371866 (Job job113199) Fixed intermittent failure to generate thumbnail for FBX filetype. #2372117 (Job job113392) Submit index success message is no longer visible. ------------------------------------------- Updates in 2022.3 (2022.3/2368025) (2022/11/09) #2361962 (Job job112411) UI for indexing part of depot and/or changelist range. Also shows estimates of time required. #2348157 (Job job112269) OCR for images and pdfs using AWS, Google and Azure libraries. #2355261 (Job job111592 ) Suport for WEBP preview (with alpha channel for thumb and preview). #2357998 (Job job107868) Configure log4j through UI to change debug levels, add class path filters, etc. #2363158 (Job job112413) MUI migration to version 5+ #2358839 (Job job112671) Support for indexing customAttributes on DAM assets. #2366950 (Job job112510) Maintaining index of head revisions of assets. #2359409 (Job job113210) Updated p4java version to 2022.1.2350821 #2364336 (Job job113154) Updated Elasticsearch clients to version 7.17.7 #2357772 (Job job112265) Auto-install and DEB/RPM packages for ES plugin-filter. #2315263 (Job job113211) Added support for audio AIFF. #2365694 (Job job113195 ) Updated JRE to 11.0.16.1 ------------------------------------------- Bug fixes in 2022.3 #2362741 (Bug #113115) Fixed - Message has no translation. #2362615 (Bug #113112) Avoid NPE by checking index status exists. #2351716 (Bug #111382 #111936) Handling windows relative and absolute path when p4 attribute and p4 obliterate is fired from windows client. #2362669 (Bug #113113) Resolved NPE by adding null check for 'path' as this may not exist (p4 describe). #2365159 (Bug #112809) AWS with Network Load Balancer (NLB) resets Elastic Client connection. ------------------------------------------- Updates in 2022.2 Patch 1 #2335090 (Job job112160) Correct Bearer 'kid' name to 'p4search' #2334092 (Job job112080 ) Use MIME type with extension before trying to generate thumb. #2328576 (Job job111974 ) Fixing status check for AWS image rekognition. #2327544 (Job job111917 ) Skip thumbnail generation during index if the blur, thumb a preview exist. Copy the blur and auto tag data from the branched attributes and use these when creating the ContentModel for Elastic search. #2325558 (Job job111818 ) Catch RuntimeExceptions for Elastic search configuration errors. #2325073 (Job job111817 ) Catch RuntimeException on status checks. #2318367 (Job job102359 ) Configuration script for Linux installations; creates: users, groups, permissions in Perforce, generates tickets and configures the Elastic search connection. #2317537 (Job job111629 ) Show status (non-admin) if the server is badly configured. Returned as a 503 Service Unavailable ------------------------------------------- Updates in 2022.2 #2311498 (Job job111522) Upgrade Elasticsearch and PluginFilter to 7.17.5 #2307200 (Job job111071) Look for f3d resource in the installed 'lib' folder. #2306400 (Job job111514) On startup only wait for required service status. Optional service status like ES filter plugin, Extensions and AutoDetection can allow search to start if they fail. #2306318 (Job job110481) GET endpoint to install extension from lib folder. GET endpoint to list extension binaries from lib folder. #2305819 (Job job111244) Only run image detection on Video, Vector and Image thumbnails (skip audio as this would try to detect a graph). #2305744 (Job job111249) Use existing or default configuration values for Image Detection configuration. #2305576 (Job job111516) Thumbnail and preview support for vector .glb formatted 3D files. #2304973 (Job job111515) Thumbnail and preview support for vector .obj formatted 3D files. #2304968 (Job job111214) Process the indexer extension call completely before processing the asset extension call for multi-threaded, concurrent environments. #2304967 (Job job111226) Prevent duplicate key value pairs of 'blur' entry when there are multiple tags configured in P4Search. #2300505 (Job job111520) Update blur and thumb if preview changes. For the update to trigger the blur field must have been removed. #2300370 (Job job111517) Thumbnail and preview support for .au and .ogg audio formats. #2298270 (Job job110650) Thumbnail and preview support for vector .fbx formatted 3D files. #2297764 (Job job111519) Use FFmpeg libraries for audio processing if the default java libraries cannot process stream. #2297256 (Job job110918) Add AudioGraph class to quantise and draw bars for use with audio thumbnail and preview. #2296392 (Job job110891 job111212) Add Capability endpoints: GET http://localhost:1601/api/v1/capability/media GET http://localhost:1601/api/v1/capability/parse GET http://localhost:1601/api/v1/capability/image GET http://localhost:1601/api/v1/capability/video #2296117 (Job job110917) Update ContentModel to check for video and audio streams. Process thumbnail first as a Tika parse error was closing the shared tmp file. #2295630 (Job job110917) Thumbnail and preview support for video keyframe. #2284652 #2284116 (Job job110630) Add enhanced status for admin user to listing Extensions on the status page. Normal users will not see extensions. #2278780 (Job job111518) Update P4Java 2021.2.2278127 #2273555 (Job job110265) Update image processing pipeline to look for preview data and use that for thumb/blur generation. #2273198 (Job job110198) Bug fix when upgrading a list of tags. #2271188 #2269752 (Job job110127) Generate preview image and saved in the file revision attribute. Added new property com.perforce.p4search.index.preview-size=1440 for preview image generation. #2269713 (Job job110126) Log Tika detect errors and continue - saving Blur and Thumb. #2267991 #2267988 (Job job110143) Thumbnail support for SGI, TGA, BIGTIFF, WEBP, TIFF and PSD (along with lots of other types). #2261466 (Job job109442) Upgrade Tika to 2.x #2259855 (Job job109693) Support AWS Image Rekognition for automated label support. #2259032 (Job job109697) Support GoogleLabel detection for automated label support. #2258758 (Job job109838) Configuration for 'best' and 'threshold' values for automated label support. ------------------------------------------- Updates in 2022.1 #2248986 #2241888 (Job job108860, job109411) Added image recognition, auto tagging and image detection status. #2242189 (Job job109631) Upgrade Elasticsearch and PluginFilter to 7.17.0 #2241526 (Job job109599) Batch endpoint added to support Helix-Artifacts stats. #2247964 (Job job109634) P4Search terminates if ElasticSearch is not available at startup. #2241965 (Job job109632) Update p4java to 2021.2.2240592 #2247615 (Job job109633) Upgrade Tika parser to 1.28 #2248874 (Job job109635) Security update to axios library. #2244193 (Job job109636) Upgrade JRE to 11.0.14 ------------------------------------------- Updates in 2021.4 patch 3 #2226008 (job108967) Update log4j2 to version 2.17.0 CVE-2021-45105 ------------------------------------------- Updates in 2021.4 patch 2 #2221532 (job108915) Update log4j2 to version 2.16.0 CVE-2021-44228 CVE-2021-45046 ------------------------------------------- Updates in 2021.4 patch 1 #2211782 (Job job108677) Add missing protectFilter plugin versions for Elasticsearch. ------------------------------------------- Updates in 2021.4 #2208434 #2208517 (Job job108600) Upgrade Elasticsearch to 7.15.2 #2208389 (Job job108601) Extend PluginFilter to use ES 7.15.1 and 7.15.2 #2206635 (Job job108549) P4Search UI Content Search only return results in 'content' not depotPath or other fields. #2206264 (Job job108545) Change configuration behaviour to exit (with status -1) if no configuration is found. #2205626 (Job job108561) Add support for BlurHash. Uses library to encode a thumbnail into a 4x4 pixel blur. #2205306 (Job job108540) Remove logging showing JWT from INFO level logs. #2204926 (Job job108401) Update P4Java to 2021.1.2204486 to use ExtensionSummary object when installing Extensions. #2203599 (Job job108464) Use tmpFile from p4 print to generate thumb and add it to p4 attributes. Thumb is encoded as PNG 240x240 max dimensions. #2201611 (Job job108344) On Startup P4Search sets the Perforce Configuration for the P4Search external URL and X-Auth-Token used by the Lua Extensions. Added new property: com.perforce.p4search.service.external-url #2201596 (Job job108341) Extensions to read from p4 property instead of extension config. #2200862 (Job job108342) Extension Endpoints to Install, Delete and List. #2199632 (Job job108340) Fixing bug where we return bearer error when auth fails regardless of which auth method was used. #2198717 #2198244 (Job job108343) Build signed Extensions for p4search. #2198228 #2201632 #2207644 (Job job107832) Update Configuration to group properties and support multiple language for title and help text. #2197006 (Job job107847) Serverside Extensions for indexing attributes. #2196126 (Job job106953) JWT support for UI and maintain legacy BasicAuth for Swarm API access. #2191592 (Job job108062) Add i18next multiple language support to front-end. #2190888 #2198225 (Job job107833) Stats logging for HelixArtifacts, events: P4_DOWNLOAD, P4_UPLOAD, CACHE_DOWNLOAD, HA_SEARCH, HA_DETAILS #2188050 (Job job107745) Indexes Perforce file revision attributes in a new AssetModel for DAM. #2177100 #2177631 (Job job107746) Remove com.perforce.p4search.security.required and force 'admin' user access. #2174688 #2174761 (Job) Update ConnectionSession to use a CloseableConnection instead of IOptionsServer. #2173783 (Job job107727) Split Index and Service users. When upgrading we use the existing Service user for both roles. ------------------------------------------- Updates in 2021.3 patch 3 #2226006 (job108967) Update log4j2 to version 2.17.0 CVE-2021-45105 ------------------------------------------- Updates in 2021.3 patch 2 #2221530 (job108915) Update log4j2 to version 2.16.0 CVE-2021-44228 CVE-2021-45046 #2221595 (job108916) Update Gson to 2.8.9 WS-2021-0419 ------------------------------------------- Updates in 2021.3 patch 1 #2180834 (job107972) CentOS RPMs are now signed. ------------------------------------------- Updates in 2021.3 #2168286 (Job job107585) Update JRE's to 11.0.11+9 #2167625 (Job job107568) Remove Href Anchor link from status page to configuration page. #2167617 (Job job107570) Elastic Search Cardinality Aggregation gives wrong total number. Added precision_threshold to raw search query. #2164973 (CVE-2021-22145) Update elasticsearch to 7.13.4 #2164849 (Job job107581) Update P4Java #2160343 (CVE-2021-35516 CVE-2021-35517 CVE-2021-35515 CVE-2021-36090) Security fix - update org.apache.commons:commons-compress:1.20 --> 1.21 #2155722 (Job job107259) Search Help Url search-service update #2155382 (Job job107177) Setting p4search extended filespec parser to use new p4java filespec parser. #2155360 (CVE-2017-18640) Updated ElasticSearch High Level Rest Client to 7.13.3 due to Snake YAML version. #2154796 (Job job107234) fixed the bug of Search UI query results count incorrect. ------------------------------------------- Updates in 2021.2 #2134895 (Job #106784) New end-point to purge ElasticSearch entries during an obliterate. #2081568 (Job #105517) Full support for ElasticSearch queries with /api/v1.1/search/raw endpoint. #2133019 (Job #106892) Add 'Date' field to 'content' and 'revision' models. #2100953 (Job #105935) ThreadPoolExecutor leak JDK-6399443