Errore durante l'esecuzione di uno script Python su AWS Lambda

Aug 21 2020

Sto cercando di eseguire lo script Python seguente su AWS Lambda, che ho eseguito manualmente e ho potuto ottenere il risultato sul mio bucket S3 di output senza alcun problema. Ma ora, quando richiamo lo script da AWS Lambda ottenendo l'errore seguente, non sono sicuro se mi manca qualcosa nello script?

#!/usr/bin/env python3
import boto3

#Function for executing athena queries
def run_query(Event, context):
    ...
    run_query(query, database, s3_output)
    client = boto3.client('athena')
    response = client.start_query_execution(
        QueryString=query,
        QueryExecutionContext={
            'Database': 's3_accesslog'
            },
        ResultConfiguration={
            'OutputLocation': s3_output,
            }
        )
    
#import datetime 
import datetime
year = datetime.date.today()
year = year.strftime("%Y")
month = datetime.date.today()
month = month.strftime("%m")
day = datetime.date.today()
day = day.strftime("%d")

#select bucket 
s3_input = "s3://smathena/cf-ant-prod/year=%s/month=%s/day=%s" % (year, month, day)
   
#Athena configuration
s3_ouput = 's3://smathena/athenatest/'
database = 's3_accesslog'
table = 'test_output1'

#Athena database and table definition
create_database = "CREATE DATABASE IF NOT EXISTS %s;" % (database)
delete_table = "drop table %s.%s;" % (database, table)
create_table = \
  """CREATE EXTERNAL TABLE IF NOT EXISTS %s.%s (
  `Date` DATE,
   Time STRING,
   Location STRING,
   SCBytes BIGINT,
   RequestIP STRING,
   Method STRING,
   Host STRING,
   Uri STRING,
   Status INT,
   Referrer STRING,
   UserAgent STRING,
   UriQS STRING,
   Cookie STRING,
   ResultType STRING,
   RequestId STRING,
   HostHeader STRING,
   Protocol STRING,
   CSBytes BIGINT,
   TimeTaken FLOAT,
   XForwardFor STRING,
   SSLProtocol STRING,
   SSLCipher STRING,
   ResponseResultType STRING,
   CSProtocolVersion STRING,
   FleStatus STRING,
   FleEncryptedFields INT,
   CPort INT,
   TimeToFirstByte FLOAT,
   XEdgeDetailedResult STRING,
   ScContent STRING,
   ScContentLen BIGINT,
   ScRangeStart BIGINT,
   ScRangeEnd BIGINT
   )
   ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
   LOCATION '%s'
   TBLPROPERTIES ('skip.header.line.count' = '2');""" % (database, table, s3_input)

#Query definitions
query_1 = "SELECT * FROM %s.%s where CAST(status AS VARCHAR) = '404';" % (database, table)

#Execute all queries
queries = [ create_database, delete_table, create_table, query_1 ]
for q in queries:
   print("Executing query: %s" % (q))
   res = run_query(q, database, s3_ouput)

Ma ora, quando richiamo lo script da AWS Lambda ottenendo l'errore seguente, non sono sicuro se mi manca qualcosa nello script?

{
  "errorMessage": "run_query() takes 2 positional arguments but 3 were given",
  "errorType": "TypeError",
  "stackTrace": [
    "  File \"/var/lang/lib/python3.7/imp.py\", line 234, in load_module\n    return load_source(name, filename, file)\n",
    "  File \"/var/lang/lib/python3.7/imp.py\", line 171, in load_source\n    module = _load(spec)\n",
    "  File \"<frozen importlib._bootstrap>\", line 696, in _load\n",
    "  File \"<frozen importlib._bootstrap>\", line 677, in _load_unlocked\n",
    "  File \"<frozen importlib._bootstrap_external>\", line 728, in exec_module\n",
    "  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n",
    "  File \"/var/task/lambda_function.py\", line 86, in <module>\n    res = run_query(q, database, s3_ouput)\n"
  ]
}``

Risposte

2 DanielFarrell Aug 21 2020 at 19:27

la tua funzione lambda_handle,, non è conforme all'interfaccia lambda di python :

def handler_name(event, context): 
    ...
    return some_value

gli input alla tua funzione dovrebbero essere in event. Un altro esempio da quel collegamento:

def my_handler(event, context):
    message = 'Hello {} {}!'.format(event['first_name'], 
                                    event['last_name'])  
    return { 
        'message' : message
    }  

Mi aspetterei query, databasee s3_outputdi essere parte del eventnel tuo caso. Probabilmente dovresti avere returninformazioni sulla query athena in esecuzione.