KL-001-2021-009: CyberArk Credential Provider Race Condition And Authorization Bypass

Title: CyberArk Credential Provider Race Condition And Authorization Bypass
Advisory ID: KL-001-2021-009
Publication Date: 2021.09.01
Publication URL: https://korelogic.com/Resources/Advisories/KL-001-2021-009.txt


1. Vulnerability Details

     Affected Vendor: CyberArk
     Affected Product: Application Access Manager/Credential Provider
     Affected Version: Prior to 12.1
     Platform: Linux/Windows/zOS
     CWE Classification: CWE-326: Inadequate Encryption Strength,
                         CWE-362: Concurrent Execution using Shared Resource with Improper Synchronization ('Race
Condition'),
                         CWE-923: Improper Restriction of Communication Channel to Intended Endpoints
     CVE ID: CVE-2021-31797


2. Vulnerability Description

     CyberArk's Credential Provider loopback communications on TCP
     port 18923 are encrypted with key material that has extremely
     low entropy. In all currently-known use cases, the effective
     key space is less than 2^16. For an attacker who understands the
     key derivation scheme and encryption mechanics, knowledge of the
     source port and access to the payloads of a given client-server
     exchange are sufficient to reduce effective key space to one. In
     cases where the source port is not known, the encrypted payloads
     will be unable to withstand a brute force attack.

     Additionally, the user identification mechanism used
     by CyberArk's Credential Provider is vulnerable to a race
     condition where an unauthorized/unprivileged user can submit
     one or more encrypted query requests. If the race is won,
     the attacker will be able to retrieve sensitive information
     including passwords and password metadata.


3. Technical Description

     Based on analysis and observations, the key derivation process
     for CyberArk's Credential Provider loopback communications on
     TCP port 18923 can be summarized as follows:

       - start a SHA1 hash (Hash1)
       - update Hash1 with the decimal representation of the source port
       - update Hash1 with an undocumented, hard-coded byte sequence
       - finalize Hash1
       - construct encryption key using Hash1[0:16]

     Capturing loopback communications (e.g., with a sniffer)
     requires elevated privilege. Thus, the risk associated with
     this attack vector can be partially mitigated through basic
     system hardening.

     However, access to loopback communications is not the only
     method by which the Credential Provider can be attacked. An
     unprivileged user can simply open a connection, submit an
     encrypted request, and if the conditions are right, the
     Credential Provider may be "tricked" into generating a valid
     response.

     Below, a summary of two distinct races is given. Each race
     consists of two query requests: one made by an authorized user
     (u_auth) and the other made by an unauthorized/unprivileged user
     (u_unauth). Observe (see details provided below) that only the
     authorized query is satisfied in the first race, while both
     queries are satisfied in the second race. By piggybacking on a
     lock file created during u_auth's request, the user controlling
     u_unauth's account is able to retrieve information s/he is
     not authorized to view/possess. Clearly, this is a security
     breach. Factors that make this attack possible include:

       - The key material generated for these communications is
       weak. The effective key space is less than 2^16 because
       the key is derived from the TCP source port used to make
       the request and an undocumented, hard-coded byte sequence
       embedded in the key derivation code (henceforth referred to
       as Suffix1).

       - The user identification mechanism used by the Credential
       Provider is vulnerable to a race condition where a shared,
       ephemeral lock file exists in a common folder from the
       time that the client makes its request to the time that the
       Credential Provider's response is received/processed. The
       window of time in which this file persists affords an
       unauthorized user the opportunity to submit one or more
       distinct requests that subsequently induce the provider
       to reuse the yet-to-be-removed lock file. At that point,
       pending requests (piggyback requests) will be deemed to have
       originated from the same user as the first request (original
       request), and if that user is authorized to make each request,
       then each will be satisfied by the Credential Provider so
       long as the lock file persists. An attacker who understands
       how this mechanism works, can simply wait for a lock file
       to come into existence (or predict its existence based
       on process table monitoring and TCP port allocation). Once
       detected (or predicted), a piggyback request can be submitted
       provided that it originates from the "same" source port as
       the original request.

       - Requests originating from loopback addresses other than
       127.0.0.1 (e.g., 127.0.0.{2,3,4}, etc.) are honored by the
       Credential Provider. This makes it possible for an attacker
       to make a piggyback request from the "same" source port
       as the original request. Without the SO_REUSEPORT socket
       option and cooperating processes, each source address/port
       tuple must be unique on a given system. In other words,
       two non-cooperating processes can't simultaneously bind to
       the same source address/port (e.g., 127.0.0.1:10000). One or
       the other will be rejected by the operating system. However,
       since 127.0.0.1:10000 and 127.0.0.2:10000 are distinct tuples,
       they are viewed as distinct endpoints (by the operating
       system) and are therefore allowed to coexist. The Credential
       Provider, upon receiving a request from either endpoint,
       ignores (or discards) the source address, and incorrectly
       associates that communication as having originated from the
       "same" port (i.e., port 10000 per the example at hand).

       - Note that if the Credential Provider is unable to
       communicate with the Vault, it will continue to answer cached
       queries. If log records are world readable, they can reveal
       past queries that were both legitimate and successful. If
       the provider is configured to maintain a cache, those query
       results are likely still resident in the cache. Thus, an
       attacker may use knowledge of cache behavior and available
       log records to construct piggyback requests that are likely
       to be satisfied when the race is won.

     Below, a sanitized excerpt of the authorization policy
     recovered from the Credential Provider's local cache
     (appprovider_cache.dat) on [NAME REDACTED] is shown. Observe
     that u_auth is listed as an authorized user. The scope of this
     authorization is not fully known. Through testing, it was
     determined that u_auth is authorized to make password queries
     from at least two distinct safes (AIM_SAFE1 and AIM_SAFE2).

       --- policy ---
       <?xml version="1.0" encoding="UTF-8"?>
       <AppMetaData>
         <AIM>
           <AuthMechanism Type="AppID" />
             <AuthReq LastID="142">
               <IPAuth>
                 <IPReqData ReqVal="10.10.1.11" ID="11" />
                 <IPReqData ReqVal="10.10.1.12" ID="12" />
                 ...
                 <IPReqData ReqVal="10.10.1.140" ID="140" />
               </IPAuth>
               <OSUserAuth>
                 ...
                 <OSUserReqData ReqVal="u_auth" ID="6" />
                 ...
               </OSUserAuth>
             </AuthReq>
           <ExtendedApplicationRestrictions isEnable="True" />
         </AIM>
       </AppMetaData>
       --- policy ---

     Below, the result of an authorized query made by u_auth to
     retrieve app_123's password located in the AIM_SAFE1 safe
     is shown. Note that the CyberArk "clipasswordsdk" utility
     transparently encrypts the request and decrypts the response.

       $ clipasswordsdk GetPassword -p AppDescs.AppID=APP_ID -p Query="Address=enco;TokenID=APP123Pass;Env=env1" -o Password
       --- output ---
       [PASSWORD REDACTED]
       --- output ---

     Below, the result of two unauthorized queries made by u_unauth
     are shown. These queries attempt to retrieve app_456's password
     located in the AIM_SAFE2 safe. Note that these responses
     are essentially what an observer could capture if sniffing
     the loopback interface. Also note that the Elements field
     contains encrypted query results. Once decrypted (as shown
     farther below), it is evident that the first response contains
     an error message indicating that the query failed. Hence,
     that race was lost. However, the second response clearly
     contains valid data. Hence, that race was won. This implies
     that no single race is a sure bet. Rather, factors such as
     current system load, number of processors, processor speed,
     etc. will influence the outcome of any given race. Note that
     an unconstrained user may be able to influence system load
     sufficiently to consistently win the race.

       --- RESPONSE_1_BEGIN [TCP_PORT=44222] ---
       <?xml version="1.0" encoding="UTF-8" standalone="no" ?>
       <Message>
         <Header>
           <ProtocolVersion>995000</ProtocolVersion>
           <ProtocolInfo>3</ProtocolInfo>
         </Header>
        
<Elements>9232456D7707CEBCC280184CB311F0871521DBFF0546A5E858974FE9E4AC8B4260FB4C579ACFB603905D1CFD1CA8EF76CD9AC32AE6C7CD6EC85FA57BD3E035514EEB72E767FBEA3FF2F4FA45D6F3D7E34B0DDACF9E2A3443E43C24D84544534A65046A6A4CCBED7CF6C72733F7B05CFD2F805E51B277100E1A9D42DA8D759EE6DE43F0D26ED41750EE428F82481CC96C8A3177C09F8A882138C294A4ADBDC270DE55128ABDCFCD6A5451A692DF4AE4035CB8D23C38176E78669903C680C89C14B00798568718FECAA1B7143ABAEEE612CE394DA4B7BD550338DB319BE607EED7F57EBCA05AA52CFAC03B02A4EC49F8F6B9E60375C32CEAA963017F590E3FBBAF4652664FC79AB355CB23EF50581A76FFC207731975FAF04CAAFD32F10B508E16660F0BCF6A55FDF3C2322110FBA425CFD06530B373B493957EF76E62100788D9FF1954BD12CBDD9D4E0393D3F6F411BD28AC788A55D6291BCDC439D895AA24E3A307A479EF420524AA3131C137F2EB9F</Elements>
       </Message>
       --- RESPONSE_1_END [TCP_PORT=44222] ---

       --- RESPONSE_2_BEGIN [TCP_PORT=44228] ---
       <?xml version="1.0" encoding="UTF-8" standalone="no" ?>
       <Message>
         <Header>
           <ProtocolVersion>995000</ProtocolVersion>
           <ProtocolInfo>3</ProtocolInfo>
         </Header>
        
<Elements>3681C9C1467C74EC3FFB7982B5E2639602D87E70D4E00EDCD6A65AFAF2093077B7CA27126C26A6F23DAEFB90287E9CDF99C20ED9F1835BC849855F8B7CFA437F72C8EF57EA077FA14DA625FAE93C0AAA0883FC130DA386E03622C80439A2F35F6508D08218E8ABE0D2000A2944908D13B5AE20854C4D8DCF30E790E25D6C4642BE52C6BA494FDDAB5A422C644C1D4A4688A9E9450F681BFD0949AB1A3358A355DFA797F2D416A9B1C2EF4C960A2A0B4C2F6F278B0B705D29F3DC3CFC9DA79B43B341D1AD8C45F602796CF627D5721DE7278B984E59CA5B5364A62CEFFAF0EDB9E7E33DCE349781AD0934DDD58E0A27EE669879D4641170C74FF237AD69C2F2C74263F48433213952F3E436EAFA65A69E12833EE189416F5F29A7A420D284180ED3C37D58DBE06600E4F3E01FF13C8AC98A101396E2FB0C382CF2C44C5E9487A31BE8665C2C6CC03F8982E719928537F2446068A849397A7132245F5521C990CA037C9C88C707F661CDD688AC9C45CC23A4B508EEE5E858E47E1C0BC31843BE43522F80DD29F0D6CD4233622D1F95A563C5425DEC73DBD47655525DCA19BBE64C177F8A0462D073FE99F135F2C056108D1F3D7B91CF055681DE5FBB12ABC92C08F872FACCE86C703B2F5CBFD49DB85DFBB484AB4C9308E804053F96D73E2DFD79D1EC55AE26358F4C02D33D0719300627B336B4FF99C294E0A713B6D33381DAB0241448A409E8346AA35F8352E1B0851BC1338EA6863E49DDB98B8CAB8EBB566C257F079BA16B6C28C4F1BA590B61B8F301E941867D3162231016DAA0C738C64C7B4DF355A8C8FCB08B866B81D9C23B9CAB626FFE53601D114D04E370D085B97473E8EE56DE4B7083D645C7E11CC51EF8BEA72A424BFD7FE3DCB6DF2EB58D95D907347627E450E83DD8CFEDA1F438953BFFAF52BEE65A3980F50DED183D6AFA6F0CD1C4B4F28A0FC16540055B76E063015D13100BE3492B9B32C02F45819506B39247CF4CA65CF04BE4BCDCF38EB4E3F44696299AFF93177E9F3E0B8ABD14BF51C35E3493B604173E83B6C00E85B580639A57B961A340881738D925068817F15E5FA565C95B8E913E200F82E21AA51C23C751AD75C3CE32C50B3BCF7B10C680A7CBA7356D2A8027D138AC6B0294AD760F1C71BA7DA8F53B997E60B185998A72693C3A88C43817648D285BFAFCC9F26CD3C37B96862BCED33EB48E536EDA29DBB1E0D7EC5680478BD99361D2C04B5F9E079EE8DC0D2CD76362507C8C13759156D450F1EE283171D3A5D9C8C3B9D284D3BF3B7729529E0FACC3424B27C1A62392B9A127AAF98982318459F20B5A1C60B9DB4AAA1F9AD0F2EEE373E46DC0231FF106</Elements>
       </Message>
       --- RESPONSE_2_END [TCP_PORT=44228] ---

       --- DECRYPTED_ELEMENTS_RESPONSE_1_BEGIN [TCP_PORT=44222] ---
       <Elements>
         <Element>
           <ElementType>ErrorResponse</ElementType>
           <ResponseBase>
             <ResponseId>1</ResponseId>
             <ErrorResponse>
               <ErrorCode>-1</ErrorCode>
               <ErrorMsg>APPAP087E Application authentication failure</ErrorMsg>
             </ErrorResponse>
           </ResponseBase>
         </Element>
       </Elements>
       --- DECRYPTED_ELEMENTS_RESPONSE_1_END [TCP_PORT=44222] ---

       --- DECRYPTED_ELEMENTS_RESPONSE_2_BEGIN [TCP_PORT=44228] ---
       <Elements>
         <Element>
           <ElementType>PasswordResponse</ElementType>
           <ResponseBase>
             <ResponseId>1</ResponseId>
             <PasswordResponse>
               <Password>REDACTED</Password>
               <Flags>0</Flags>
               <AlterPassword></AlterPassword>
               <PassProps>
                 <Address>appprodapplication</Address>
                 <ApplicationName>APP</ApplicationName>
                 <ClassType>Class H</ClassType>
                 <CreationMethod>PVWA</CreationMethod>
                 <Description>SR0118365</Description>
                 <DeviceType>Operating System</DeviceType>
                 <Env>all-prod</Env>
                 <PolicyID>H_SRV_GENERIC_Q_XXX_APC</PolicyID>
                 <RegSCI>ISCI</RegSCI>
                 <TokenID>APPPassword</TokenID>
                 <UserName>app_456</UserName>
               </PassProps>
               <PasswordChangeInProcess>false</PasswordChangeInProcess>
             </PasswordResponse>
           </ResponseBase>
         </Element>
       </Elements>
       --- DECRYPTED_ELEMENTS_RESPONSE_2_END [TCP_PORT=44228] ---

     Below, the relevant log entries generated for queries made
     during the first race are shown. Note how all timestamps
     are identical. Note also that the reason the race was lost is
     revealed by the first record in APPConsole.log. Essentially, the
     authorized query (first entry in APPAudit.log) was satisfied
     and its associated lock file (/tmp/AIM44222) was removed
     before the unauthorized query (second entry in APPAudit.log)
     could be processed by the Credential Provider. And since the
     lock file was removed, the provider had no means to look up
     the process ID of the requesting process.

       --- APPAudit.log ---
       [DATE | 18:09:08] |  ::  | APPAU001I Provider Prov_[REDACTED] has successfully fetched password
[safe=AIM_SAFE1,folder=Root,name=[REDACTED]] with query [Address=enco;TokenID=APP123Pass;Env=env1] for application
[APP_ID]. Fetch reason: []
       [DATE | 18:09:08] |  ::  | APPAU002E Provider Prov_[REDACTED] has failed to fetch password with query
[Address=appprodapplication;Env=all-prod;TokenID=APPPassword] for application [APP_ID]. Fetch reason: []. Failure
reason: [APPAP087E Application authentication failure]
       --- APPAudit.log ---

       --- APPConsole.log ---
       [DATE | 18:09:08] |  ::  | APPAP087E Application authentication failure for Application APP_ID (CASCU086E Failed
to find application process id. (Error: The lock file (/tmp/AIM44222) couldn't be opened, Error code: 2).)
       [DATE | 18:09:08] |  ::  | APPAP002E Provider Prov_[REDACTED] has failed to fetch password with query
[Address=appprodapplication;Env=all-prod;TokenID=APPPassword] for application [APP_ID]. Fetch reason: []. Failure
reason: [APPAP087E Application authentication failure]
       --- APPConsole.log ---

     Below, the relevant log entries generated for queries made
     during the second race are shown. Note how all timestamps are
     identical. The authorized query corresponds to the first entry
     in APPAudit.log, and the unauthorized query corresponds to the
     second entry. Note how two distinct safes were queried. This
     suggests that safes not normally accessed/queried by a given
     Credential Provider may be targeted by an attacker from
     a different system. Clients should be cautioned that the
     contents of any given safe should be confined to a single
     security domain.

       --- APPAudit.log ---
       [DATE | 18:09:13] |  ::  | APPAU001I Provider Prov_[REDACTED] has successfully fetched password
[safe=AIM_SAFE1,folder=Root,name=[REDACTED]] with query [Address=enco;TokenID=APP123Pass;Env=env1] for application
[APP_ID]. Fetch reason: []
       [DATE | 18:09:13] |  ::  | APPAU001I Provider Prov_[REDACTED] has successfully fetched password
[safe=AIM_SAFE2,folder=Root,name=[REDACTED]] with query [Address=appprodapplication;Env=all-prod;TokenID=APPPassword]
for application [APP_ID]. Fetch reason: []
       --- APPAudit.log ---


4. Mitigation and Remediation Recommendation

     The vendor has released an updated version (v12.1) which
     remediates the described vulnerability. Release notes are
     available at:

    
https://docs.cyberark.com/Product-Doc/OnlineHelp/PAS/Latest/en/Content/Release%20Notes/RN-WhatsNew12-1-CPs.htm?tocpath=Get%20Started%7CWhat%E2%80%99s%20New%7CRelease%20Notes%7C_____4


5. Credit

     This vulnerability was discovered by Klayton Monroe of
     KoreLogic, Inc.


6. Disclosure Timeline

     2020.11.04 - KoreLogic submits vulnerability details to
                  CyberArk.
     2020.11.05 - CyberArk acknowledges receipt and the intention
                  to investigate.
     2020.11.16 - KoreLogic and CyberArk meet to discuss the
                  details of this and other reported
                  vulnerabilities. Both parties agree that the
                  remediation timeline will extend significantly
                  longer than the standard 45 business days specified
                  in the KoreLogic Public Disclosure Policy.
     2021.01.14 - 45 business days have elapsed since the
                  vulnerability was reported to CyberArk.
     2021.01.21 - KoreLogic and CyberArk meet to discuss proposed
                  remediation efforts for this and other reported
                  vulnerabilities.
     2021.03.24 - 90 business days have elapsed since the
                  vulnerability was reported to CyberArk.
     2021.04.22 - CyberArk notifies KoreLogic that the reported
                  vulnerability will be mitigated in a version
                  scheduled for release in late May, 2021.
     2021.05.10 - 120 business days have elapsed since the
                  vulnerability was reported to CyberArk.
     2021.05.10 - CyberArk provides KoreLogic with the CVE for this
                  vulnerability. Vendor requests KoreLogic delay
                  public disclosure until the end of June, 2021.
     2021.06.08 - KoreLogic and CyberArk meet to discuss the details
                  of the product release and revisit timeline for
                  public disclosure. CyberArk informs KoreLogic that
                  the Linux/Windows version of the Credential
                  Provider will be released at the end of June, 2021.
                  A Credential Provider for the zOS platform will be
                  released at the end of July, 2021. KoreLogic agrees
                  to delay public disclosure of this and other
                  reported vulnerabilities until 2021.08.15.
     2021.06.23 - CyberArk releases Credential Provider v12.1 for
                  Linux/Windows platforms.
     2021.08.05 - 180 business days have elapsed since the
                  vulnerability was reported to CyberArk.
     2021.08.10 - CyberArk informs KoreLogic that the zOS Credential
                  Provider update has been released to their
                  customers. Requests that KoreLogic forgo
                  publication of the Proof of Concept code as an
                  unforseen issue prevents some customers from
                  updating in the near term.
     2021.08.27 - KoreLogic suggests delaying the release of the
                  Proof of Concept until a to-be-determined future
                  date.
     2021.08.30 - CyberArk tenders 2022.01.01 release date for the
                  Proof of Concept.
     2021.09.01 - KoreLogic public disclosure.


7. Proof of Concept

     At the vendor's request, KoreLogic has agreed to delay
     publication of the Proof of Concept while customers continue
     to deploy the updated versions of the product.



The contents of this advisory are copyright(c) 2021
KoreLogic, Inc. and are licensed under a Creative Commons
Attribution Share-Alike 4.0 (United States) License:
http://creativecommons.org/licenses/by-sa/4.0/

KoreLogic, Inc. is a founder-owned and operated company with a
proven track record of providing security services to entities
ranging from Fortune 500 to small and mid-sized companies. We
are a highly skilled team of senior security consultants doing
by-hand security assessments for the most important networks in
the U.S. and around the world. We are also developers of various
tools and resources aimed at helping the security community.
https://www.korelogic.com/about-korelogic.html

Our public vulnerability disclosure policy is available at:
https://korelogic.com/KoreLogic-Public-Vulnerability-Disclosure-Policy.v2.3.txt