How to bypass waf upload forms from RFC specification
Background
Traditional waf is based on rule matching, if it just uses the rules to match the whole packet without any difference, when the number of rules becomes more and more, it will cause more performance loss, and of course, false alarms will also occur. In order to solve these problems, it is necessary to parse the packets and match the rules with precise locations.
Normal business upload form is commonly used, not only to pass parameters, but also for file uploads, of course, this is also a good attack point, waf want to be able to accurately intercept attacks on the form, you need to multipart/form-data format data parsing, and for each part, such as parameter values, file name, file content for the targeted rules to match the interception.
Although RFC standardizes the format and parsing of multipart/form-data, due to the different implementation mechanisms of different back-end programs, and the additions of RFC documents, the parsing methods are different. For waf, it is difficult to customize the parsing for each backend program especially for cloud waf.
So this paper mainly discusses, using waf and the back-end program on the multipart/form-data parsing differences, resulting in waf bypass. multipart/form-data phase 䏌 RFC: 䏌 RFC: 䏌 RFC: 䏌 RFC: 䏌 WAP.
This form data contains a file, name as name, filename as file.jsp, file_content as This_is_file_content., and a non-file parameter, whose name is key and value is This_is_a_value.
For the upload form type, the Content-Type must be multipart/form-data, followed by a boundary parameter key-value pair, which is used to separate parts of the form.
If multipart/form-data is written incorrectly, or without a boundary, then the backend will not be able to accurately parse every detail of the form.
2. Boundary
boundary: RFC2046
boundary needs to follow the following BNF Bacchus paradigm
boundary bchars bcharsnospace
bchars := bcharsnospace /" "
Simple explanation is that, boundary can not end with a space, but all other positions can be a space, and the length of the character between 1-70, the syntax of this rule applies to all types of multipart, of course, not all programs in accordance with the provisions of this multipart parsing.
From the previous introduction of the multipart basic format can be seen, the real boundary between the various parts of the form as a boundary is not only the value of the boundary in the Content-Type, the real boundary is the value of --- and boundary and the end of the CRLF composed of the separation line, of course, in order to accurately analyze the data of the various parts of the form, you need to make sure that the separation line Of course, in order to accurately parse the data in each part of the form, it is necessary to ensure that the separator line does not appear in the normal form of the contents of the file or parameter values, so the RFC also suggests the use of specific algorithms to generate the boundary value.
flask parsing results
There are two points to note here, first, the last delimited boundary of the final form data should end with -. Second, the RFC states that the original text is
The Content-Type field for multipart entities requires one parameter, "boundary". The boundary delimiter line is then defined as a line consisting entirely of two hyphen characters ("-", decimal value 45) followed by the boundary parameter value from the Content-Type header field, optional linear whitespace, and a terminating CRLF.
That is, the overall separating boundary can contain OPTIONAL linear whitespace.
blank space
Note: This article uses space can be used instead, the article only introduces the results of the use of space, you can test the other, waf or back-end program in the parsing of , will produce a lot of different results, interested in their own test.
First use the value of the boundary followed by a space to test, flask and php are able to parse out the form content.
php parse result
Although the value of the boundary is followed by a space, but as a separator line without a space can also be parsed normally, but after testing found that if in accordance with the RFC rules directly in the separator line to add a space, the effect will be different.
For flask is in accordance with the RFC regulations to achieve, regardless of Content-Type in the value of the boundary is not added after the space or add any space, in the form of non-terminal line can be added to the space, do not affect the parsing of form data, but need to pay attention to is that, in the end of the last line, add space will lead to parsing failures.
It is interesting to note that during php parsing, you can't add a space to a non-terminated line, but adding a space to a terminated line does not affect parsing.
As you can see, the file content data in the separator line with spaces was not parsed correctly, while the non-file parameters without spaces were parsed successfully, and spaces were added to the end of the separator line.
When testing, I stumbled upon that if you put a space between multipart/form-data and ;, such as Content-Type: multipart/formdata; boundary="I_am_a_boundary", flask will fail to parse it, but php will parse it normally.
"jorn" :null,
"urigin": "t"t:///mmm example, con: 8081/post"
Normally, a flask that matches parses by regularity would not do this, as implemented in werkzeug/http.py:L406.
Simply put is the Content-Type: multipart/form-data; boundary="I_am_a_boundary" for regular matching , and then the first set of matching results as a mimetype , the second group as a rest, by the back of the processing boundary value, look at this regular.
_option_header_start_mime_type re.compile ?")
To look at it beautifully, use regex101 to look at it.
TEST STRING
multipart/form-data ; boundary="I_am_a"_boundary"
Obviously, since the first group matches non-null characters, it stops at the space, but the second group must be , null, it can't get the boundary, and eventually the parsing fails.
double quote
The value of the boundary is supported to be written in double quotes, like the value of a parameter in a form, so that when writing a delimited line, the contents of the double quotes can be used as the value of the boundary, which is supported by both php and flask. You can't do this with single quotes, which is in line with the BNF Barkos paradigm of bcharsnospace mentioned above.
Test what happens when you allow multiple double quotes to be repeated, or contain unclosed double quotes or add other characters before or after double quotes.
For php is relatively simple, because as long as the first character is not double quotes, even if it is a space, will be used as part of the boundary, so the first four parsing is similar, when the first character for the double quotes, it will be looking for the corresponding closed double quotes, if you find it, then it will be ignored after the contents of the double quotes directly take the contents of the boundary as the value.
However, if the closing double quotes are not found, the boundary fetch fails to parse multipart/form-data.
Of course, in the last case, it will take an empty boundary value, and I thought it would fail to parse, but funny enough, the boundary value is empty, and php parses it just fine, or you can just write it as Content-Type: multipart/form-data; boundary=.
Content-length: 288 ix in
In in in is file ontent. ir
18_file_content. ir in
Content-Di spositi ion: form-data: name="key2": is
Most wafs should consider this a non-compliant boundary, which causes parsing multipart/form-data to fail, so this way of bypassing wafs seems even more brute-force.
For flask, look at the regular werkzeug/http.py:L79 for parsing boundaries.
This regular can explain most of the flask parsing results in this article, here to see the flask for the boundary on both sides of the space is done to deal with, for the handling of double quotes, will take the first pair of double quotes as the value of the boundary, for the non-closed double quotes, it will be dealt with as a token form, will be quoted as part of the boundary, and will not fail as php parsing the boundary as well. boundary as part of the double quotes, and will not fail to resolve the boundary like php.
As you can see from the above rules, for the last Content-Type case, flask will also take the null value as the value of the boundary, but this will not be the same as flask's regular validation of the boundary, resulting in a failure to take the value of the boundary, which cannot be parsed, as will be mentioned in the following section.
escape sign
To distinguish between quoted string and token in flask's regular, we test how the position of the two escapes affects the parsing.
For flask, in the case of double quotes, werkzeug/http.py:L431 calls a handler that takes the content between the double quotes as the value of the boundary.
As you can see, there is also a value done after fetching the boundary value. replace("
", "
"),replace("
"', '"') operation, which recognizes the escape character as an escape, rather than a single character, so that the final value of the boundary is
I_am_a"_boundary。
For php, still the same as the token type of the boundary processing mechanism, identify \ is just a character, does not have the role of escaping, so in accordance with the double quotes mentioned above, as the second double quotes will be encountered directly close the double quotes, ignoring the content of the latter, and ultimately php will take the I_am_a \ as the value of the boundary.
Spaces & double quotes
As mentioned above, the use of spaces affects parsing, since you can use double quotes to specify the value of a boundary, how will the backend parse it if you add spaces outside or inside the double quotes?
outside double quote
For flask, it is still the same as the normal parsing without double quotes, it will ignore the space outside the double quotes (both sides), and take the content inside the double quotes as the value of the boundary, php has the same mechanism as flask when there is a space after the double quotes, but when there is a space in front of the double quotes, it will not be able to parse the content of the form data properly.
The parsing will be the same as the implementation without double quotes, in this case php will take the leading space and the trailing double quotes and the contents of the double quotes as a whole, as the value of the boundary, of course, although this conforms to the RFC rule that the boundary can start with a space, but taking the double quotes as a part of the boundary does not conform to it.
inside double quote
php will take all the content within double quotes (not double quotes) as the value of the boundary, whether it starts or ends with any space, and the number of spaces before and after the boundary in the delimited line, and the number of spaces before and after the boundary in double quotes in the Content-Type is the same, otherwise the parsing fails.
It is worth noting that, flask parsing, if the double quotes within the value of the basic to the beginning of the space, then in the separated lines similar to php as long as the number of spaces consistent, can be successfully parsed, but if the double quotes within the value of the basic to the end of the space, regardless of the number of spaces consistent, can not be parsed properly.
Hirp/1.1 200 or
".
To see why this is happening, you have to look at how werkzeug implements it, and flask's validation of the boundaries can be seen at werkzeug/formparser.py:L46.
This regular is to verify the validity of the boundary, more in line with the provisions of the RFC, only in the length of the restrictions on the smaller, can be the beginning of the space, can not end with a space, but with the use of the full match is not, so with the end of the space will also be verified.
The above figure uses boundary="I_am_a_boundary ", so the value of the boundary is "I_am_a_boundary" within double quotes, and this value will also pass the boundary regularity of the validation, but ultimately the parsing still fails, it is very strange. As mentioned in the space above, for flask, you can add any space after the boundary in the separator line without affecting the final parsing.
The reason is that when parsing multipart/form-data specific content, in order to find the split line, each line of data will be a line.strip() operation, which will remove the CRLF, of course, will be the end of all the spaces to strip away, so when the boundary does not end with a space, in the separate line can be arbitrary at the end of the space. So when the boundary does not end with a space, you can add spaces at the end of the separator line. But this will also lead to a problem, when not in accordance with the RFC provisions, with the end of the space as the value of the boundary, although through the flask of the boundary regular validation, but in the parsing of the body, but the end of the space will be strip off, resulting in the body after the processing of the separator line becomes -- I_am_a_boundary, which is the same as the This is not consistent with the boundary value obtained from the Content-Type (which contains spaces at the end), so the parsing fails because the separator line cannot be found.
End separator line
In the above space content mentioned, php in the end of the split line after the boundary to add space does not affect the final parse, in fact, is not a space problem, after testing found that, in fact, php does not take the end of the split line at all seriously.
As you can see, there is no closing separator line, php will separate each part of the form according to each separator line, and according to the Content-Length to get the value of the last part of the form, but this is extremely disrespectful of the RFC rules, generally waf will regard this kind of no closing separator line as the wrong multipart/form-data format, which will cause the overall body parsing to fail, then waf can be bypassed. This will cause the overall body parsing failure, then waf can be bypassed.
As mentioned above, flask will strip each line of multipart/form-data, but since the ending separator line needs to end with --, it will only strip the CRLF, but when parsing the boundary, the boundary can't end with a space, which will eventually lead to The end of the separator line is strict - BOUNDARY - CRLF, of course, if the use of double quotes to make the boundary to the end of the space, then the end of the separator line can be parsed correctly, but not the end of the separator line can not be parsed or will lead to the overall parsing failure.
(sth. or sb) else
As you can see from the flask code, the quoted string form of parameter names is supported, that is, the parameter names are inside double quotes.
For Java, case-insensitive writing of parameter names is supported.
3. Content-Disposition
For multipart/form-data type data, each part separated by a separator line must contain a Content-Disposition, which is of type form-data, and must contain a name parameter, such as Content-Disposition: form-data; name=" name", if this part is a file type, can be followed by a filename parameter, of course, filename parameter is optional.
blank space
Those who often deal with waf know, any space, may happen to the strange effect. For the Content-Disposition parameter, the test is to add any space to the four positions.
For the first three types, php and flask parsing are accurate.
Response
Content-Type: text/htn1; charset=rाT-8
input type="text" nane="text" val
Sbutton type="subnit">
Subnit
file.php"
sitere:t)
rray i
But the fourth one, for Content-Disposition: form-data;, php parses it accurately, and assumes it's normal multipart/form-data data, but flask fails to parse it, and returns 500 (:
The way flask handles Content-Disposition is the same as the Content-Type in request_header, after ? " match, the subsequent name and filename cannot be parsed due to spaces, except that in this case 500 is returned. For the subsequent name and filename parsing is also the same as request_header in Content-Type, the group in the latter match as rest for subsequent regular matching, match the regular, is the above part 2 (Boundary) double quotes in the _option_header _ piece_re.
Between the parameter name and the equal sign
flask parses normally
php parsing failure, not only the first part of the data can not be parsed, the second part of the non-file parameters also failed to parse, it can be seen that the php parsing will be name = / filename = as a 䏌 keyword match, when found that the name = and filename = do not exist, directly no longer parsed, this is not the same with the boundary of the resolution of the , use Content-Type: multipart/form-data; boundary =I_am_a_boundary can be parsed as normal at the boundary value.
If we don't put a space between the name and the equals sign, but only put a space between the filename and the equals sign, as in ContentDisposition: form-data; name="key1"; filename = "file.txt", then php will parse this as a non-file parameter.
If waf supports this extra-space form, then it will parse this as a file type, resulting in parsing differences, and waf mistakenly treats non-file arguments as files, which may bypass some of waf's rules.
There's not much to notice, flask will parse by the exact name.
php ignores opening spaces and converts non-opening spaces to _, see php-variables for why.
repetition parameter
Repeat name/filename parameter name
Both php and flask will take the last name/filename, from the flask code, the storage of parameters using a dictionary, due to the same key=name , so the last in the parsing, encounter the same key parameters, will be overwritten by the parameter value.
This way of repeating parameter names will be combined with other ways to bypass waf in the following sections.
Repeat the name/filename parameter name and value.
Then try repeating the entire form-data section, constructing such a packet for testing.
For php, it is consistent with repeating the name/filename in the same Content-Disposition, and will pick the last part of the same name part.
For flask, with filename, it will take the first part, and non-file arguments with the same name, it will parse the two values as a list.
This is the result of the httpbin processing. In order to accurately see the results of the flask parsing, you need to see the flask parsing result directly.
request.form/request.files。
We use ImmutableMultiDict , defined in werkzeug/datastructures.py , we can see that, in the end, form and files get all the multipart data, even if they have the same key. if we use the common keys ()/values ()/item () functions, they can only get the first key because of the same key, you need to use ImmutableMultiDict. to_dict. item () functions, we can only get the value of the first key because of the same key, if you want to get all the values of the same key, you need to use ImmutableMultiDict. to_dict( ) method, and set the parameter flat=True.
httpbin is in the processing request.form, add this kind of processing, resulting in the last see two values of the list, but in the request.files processing is not to_dict.
It can be seen that different back-end programs, the implementation may be different, if the waf in the implementation, and not all the key duplicate data are parsed out, and into the waf rule matching , then the use of duplicate key, will also become a very good way to bypass the waf.
quotation mark (punct.)
As mentioned above, _option_header_piece_re is also used to parse Content-Disposition in flask, so the mechanism for name/filename is the same as for boundary, with quotes it is quoted string, without quotes it is token.
So the main analysis of php is how to deal with, first of all php in the processing of the boundary, if the space at the beginning, then the space will be used as part of the boundary even if the space after the existence of normal double quotes to close the boundary. But in Content-Disposition, the space outside the double quotes can be ignored, of course do not use double quotes, the parameter value on both sides of the space will also be ignored.
This small paragraph title quotes, and did not use double quotes as in the previous paragraph, because php not only supports double quotes, but also supports single quotes, which is very php.
Request
flask certainly does not support single quotes, the above regular can be seen, single quotes will be treated as part of the parameter value, here looked at the Java commons-fileuploadv1.2 implementation org.apache.commons.fileupload.ParameterParser.java:L76 , also does not support single quotes when parsing parameter values.
So if waf in multipart parsing does not support parameter values in single quotes, for php, this kind of payload can lead to waf parsing errors.
Those that support single quotes will parse it as "name": "key3"}, and there is no filename parameter, which is treated as a non-file parameter.
Those that don't support single quotes will parse it as "name":"'key3'", "filename":"'file_name. txt"}, treat it as a file parameter, and treat the value of the parameter after that as the contents of the file.
This inconsistency in the parsing of the waf and the back-end handler may result in the waf being bypassed.
At this point, there is still a quotation mark problem has not been resolved, that is, if there are extra quotation marks what happens, such as Content-Disposition: form-data; name="key3 "a"; filename="file_name;txt", above in the boundary of the parsing of the results have seen , name will take key3, and ignore the content, even with double quotes, then the contents of the filename can still be correctly parsed. name takes key3, and ignores the content after that, even if it contains double quotes, so the content after the filename can still be parsed correctly? Just to see flask use regular and Java/php use character parsing to bring some differences.
Look at the specific implementation of flask werkzeug/http.py:L402.
Use _option_header_piece_re match to after, will continue from the next character to continue to enter the regular matching, so the second time to enter the regular, rest for aaaa"; filename="file_name.txt", to a start can not be matched in the regular, direct exit, resulting in filename parsing and name takes key3.
Java code has been posted above, where the terminators = "; ", that is, when the double quotes, will ignore; , but when you find the closed double quotes, the fetch does not end, will continue to look for; this will lead to the fetch until the closed double quotes outside of; will stop, which is inconsistent with the php, php, although the extra double quotes after the follow-up will affect the subsequent This is inconsistent with php, which will stop at the first occurrence of the closing quotes, even though the extra double quotes will affect the subsequent filename values.
For flask/php, if the waf parsing method is different from the backend, it may also incorrectly determine file and non-file parameters, but it is difficult for Java backends to use it, because the value of name will cause the backend to not be able to get it correctly. However, this feature is still useful, and will be introduced in the following section on file extensions.
escape sign
Both php and flask support escaping symbols in parameter values, as you can see from the _option_header_piece_re above, consistent with the value of boundary, flask escapes symbols in parameter values of quoted string type with escaping effect, and in token type it is just a character , without escaping effect. in the token type is just a character , which is not escaped.
php Although in the token type, the parsing and parsing of the boundary is consistent, the escape sign has the role of escape, but in the parsing of quoted string type parsing and boundary competition is not the same, parsing the boundary, the escape sign for a character does not have the role of escape, so boundary="aa "bbb" will be parsed as aa, while in Content-Disposition, the escape character has escaping effect.
In the same way that php parses single quotes as mentioned above, there is this payload
flask/php parses this as a non-file parameter, and based on multiple duplicate name/filename parsing mechanisms, the final result is "name".
"keyз"}
If waf doesn't support parsing of escaped symbols, but simply matches double-quote closures, then the result is "name": "key3
", "filename": ""file_name.txt"}, considered as file parameter, considered the value of the parameter after that as the content of the file, resulting in parsing discrepancy, leading to possible bypass of waf.
mentioned above, php can use single quotes to take values, in single quotes and double quotes to increase the escape character in the resolution will be different, specific references to php single quotes and double quotes and the difference between the use of.
file extension
The previous article mainly put forward some mutlipart overall waf bypass, in the source station back-end parsing is normal in the case of waf parsing failure does not enter the rule matching, or waf parsing and back-end differences, to determine whether it is a file failure, resulting in the rule can not be matched, or filename parameter does not enter the waf rule matching. Whether in the CTF competition or in the actual penetration test, how to bypass the file extension is a point of concern, so this paragraph mainly introduces, in the case of waf parsing to the filename parameter, from the protocol and back-end parsing level how to bypass the file extension.
In fact, this kind of bypass on a thought, take a simple example filename="file name.php", for a normal waf to get file_name.php, found that the extension is php, and then intercept, here does not discuss the waf rule does not contain php䏌 keyword and so on the waf rule itself, we only have one goal, that is, waf parsed out the filename does not appear php䏌 keyword, and the back-end program to verify the extension, and the back-end program to validate the extension of php䏌 keyword. There is no discussion here about the waf rules not containing php䏌 keywords and so on, we only have one goal, that is, waf parses out the filename without php䏌 keywords, and the back-end program will think it is a php file when it verifies the extension.
From a variety of programs to parse the code, in order to make waf parsing problems, interference with the character in addition to the above quotes, spaces, escape characters, and :;, here is still divided into two forms of testing.
The first five cases of flask/Java parsing results are consistent, will take the whole as the value of the filename, are contained in the phpþ keyword, which also indicates that if there is a difference in the waf parsing, will be directly truncated special characters to take the value, will lead to waf is bypassed.
In the last case, flask/Java/php parsing will directly truncate, filename=file_name, so that the backend can't get it, no matter how waf parses it, it can't be bypassed.
For php, the first three will be as flask to the same, the whole as the value of filename, the fifth space type, php will truncate, and finally take filename = file_name, this easy to understand, when there is no quotes, there is a space, that is, the end of the parameter value.
Then when testing the escape symbols, there is a truncation from \, and go to after the value of the most filename value, this parsing and boundary parsing is not the same, of course, double quotes and single quotes have the same effect.
Looking at the code, I realize that php doesn't treat \ as an escape symbol, but rather, it thoughtfully treats filename as a path and takes the name of the file in the path, after all, the parameter name is filename :)
So this parsing method has nothing to do with quotes, it's just that when php parses the filename, it will take the last \ or / followed by the value as the filename.
flask parsing result is still according to the _option_header_piece_re regular, except the third kind of filename take file_name , other will take the whole value in double quotes as filename, the escape character has the role of escaping. php the third kind of parsing will also be out of the file_name, but in the fourth kind of escaping is But in the fourth escape character is escaped, so when entering the *php_ap_basename function above, there is no , so its parsing result will also be file_name". php, the use of single quotes is consistent with the analysis of the quote section above.
For Java, except for the third case, will take the whole within quotation marks as the value of filename, but the third case is very interesting, above the quotation marks part has been analyzed, Java will continue to take the value, then the final value of filename as "file_name". php".
So for Java this exception to the characteristics, usually waf will be like php/flask in the first occurrence of closed double quotes, directly take the contents of the double quotes as the value of filename, so you can bypass the detection of file extensions.
4. Content-Type(Body)
Each part MAY have an (optional) "Content-Type" header field, which defaults to "text/plain". If the contents of a file are to be sent, the file data SHOULD be labeled with an appropriate media type, if known, or "application/octet-stream".
For some wafs that do not have encoding and parsing capabilities, the waf can be bypassed by encoding the parameter values.
Charset
For Java, UTF-16 encoding can be used.
flask can use UTF-7 encoding.
Java code, will be the file and non-file parameters are used org.apache.commons, fileupload.FileItem to store, so will be decoded, and flask will be divided into form and files, and files do not use the Content-Type in the charset for decoding werkzeug/formparser.py:L564.
(sth. or sb) else
5.1.3. Parsing and Interpreting Form Data
While this specification provides guidance for the creation of multipart/form-data, parsers and interpreters should be aware of the variety of implementations. File systems differ as to whether and how they normalize Unicode names, for example. The matching of form elements to form-data parts may rely on a fuzzier match. In particular, some multipart/form-data generators might have followed the previous advice of [RFC2388] and used the "encoded-word" method of encoding non-ASCII values, as described in [RFC2047]:
Others have been known to follow [RFC2231], to send unencoded UTF-8, or even to send strings encoded in the form-charset.
For this reason, interpreting multipart/form-data (even from conforming generators) may require knowing the charset used in form encoding in cases where the _charset_field value or a charset parameter of a "text/plain" Content-Type header field is not supplied.
RFC7578 wrote some other form-data parsing , you can specify the charset through the _charset_ parameter , or use encodedword , but the test of the three programs do not do the phase þ parsing , a lot of just used in the mail .
5. Content-Transfer-Encoding
4. 8. Other "Content-" Header Fields
The multipart/form-data media type does not support any MIME header fields in parts other than Content-Type, Content-Disposition, and (in limited circumstances) Content-Transfer-Encoding. Other header fields MUST NOT be included and MUST be ignored.
RFC7578 explicitly states that only three parameter types can appear in multipart/form-data, the other types are MUST be ignored, and the third, Content-Transfer-Encoding, is actually deprecated.
4.7. Content-Transfer-Encoding Deprecated
Previously, it was recommended that senders use a Content-TransferEncoding encoding (such as "quoted-printable") for each non-ASCII part of a multipart/form-data body because that would allow use in transports that only support a "7bit" encoding. This use is deprecated for use in contexts that support binary data such as HTTP. Senders SHOULD NOT generate any parts with a Content-TransferEncoding header field.
Currently, no deployed implementations that send such bodies have been discovered.
However, it was found in the flask code that werkzeug implements this part.
The QUOTED-PRINTABLE encoding method can also be used.
